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roMyrrrFR system linked by using 

INW>RMATION IN DATA OBIRCTS 
Backyround of the Invention 
Hiding data in imagery oc audio is a technique well known to artisans in the fieid, 
5 and is termed " steganography " There are a number of diverse approaches lo, and 
applications of, steganography. A brief survey follows: 

British patent pid)Iic^on 2,196,167 to Hiom EMI discloses a system in which an 
audio recording is electrcmicaOy mixed with a marking sigrtal indicative of the owner of 
the recording, where the combinatiixi is perceptually identical to the orighial. U.S* patents 
10 4.963.998 and 3«079»648 disclose variants of this system. 

U.S. Patent 5,3 1 9,735 to Bolt, Berenak & Newman rests on the same lainciples as 
the earlier Thorn EMI publication, but additionaUy addresses psycho-acoustic masking 
issues. 

U.S. Patents 4,425,642, 4,425.661, 5,404377 and 5,473.631 to Moses disclose 
15 various syst^ns for imperceptibly embedding data into audio signals - dte latter two 
patents particularly focusing on neural netwcnk implementations and perceptual coding 
details. 

U.S. Patent 4,943,973 to AT&T discloses a system ranploying spread spectrum 
techniques for adding a low level noise signal to other data to convey auxiliary data 
20 therewith. The patent is partioilarly illustrated in the context of transmitting network 
control signals along with digitized voice signals. 

U,S. Patent 5 J6 1,210 to U.S. Philips discloses a system in which additional low- 
level quantization levels are defhted on an audio signal to convey, e.g., a copy inhibit code, 
therewiA. 

25 U.S. Patent 4,97i471 to Gross discloses a sysian intended to assist in the 

automated nwnitoring of audio (e.g. radio) signals for copyrighted materials by reference 
to ictentification signals sublimhially embedded therein. 

U.S. Patent 5,243,423 to DeJean discloses a video steganography system which 
encodes digital data {e.g. program syndication verification, copyright marking, media 

30 research, closed captioning, or like data) onto randomly selected video lines. Delean relies 
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on television sync pulses to trigger a sicffed pseudo random sequence which is XORed with 
the digital data and combined with the video. 

European application EP 581,317 discloses a system for redundantly marking 
images with rauhi-bit identification codes. Each "I " ("0^) Int of the code is manifested as 
5 a slight increase (decrease) in pixel values around a plurality of spaced apart "signature 
points." Decoding fHoceeds by computing a difference between a suspect image and the 
original, unencoded image* and checking for pixel pertuibations around the signature 
points. 

per application WO 95/14289 describes the present applicant's pric»" work m this 

10 field. 

Koraatsu et al., describe an image marking technique in their paptj "A Prq)osal on 
Digital Watermark in Document Image Communication and Its Application to Realizing a 
Signature/ Flectmnicf; and rnmrnrniicaiions in Japan. Pan 1, Vol. 73, No. 5, 1990. pp. 22- 
33. The work is somewhat difficult to follow but apparently results in a simple ycs/no 
1 5 determination of whether the watermark is present in a suspect image (e.g. a 1 bit encoded 
message). 

Others embed Ae information "in-band" {i^. fai the visible video signal itsctf). 
Examples include US. Patents 4,328,588, 4^95,950, and 5319,453; European ^plication 
441,702; and Matsni et ai, "Video-Steganography: How to Secretly Embed a Signamie in 
20 a Picture," f>;f A Inteilecttial Pmnertv Project Pmceeding.^ Januaiy 1994. VoL 1, Issne I 
pp. 187-205, 

Cdpyright marking of video and multimedia is found in "Access Control and 
Copyright Protection for Images (ACCOPO. WorkPackage 8: Watermarking," June 30, 
1995, 46 pages. TALESMAN, appears to extend certain of the ACCOPI work. 
25 A British company, Highwater FBI, Ltd., has introduced a software product vdiich 

is said to impciceptibly embed identifying information into phc^gr^hs and other 
graphical nnages. This technology is the subject of European patent applications 
9400971-9 (filed Januaiy 19, 1994), 9504221,2 (filed March 2, 1995), and 9513790.7 (filed 
July 3. 1995), the first of which has been laid open as PCT publication WO 95/20291. 
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Brief Descrimion Of The Drawings 
Fig. 1 is a simple and classic depiction of a one dimensional digital signal which is 
discretized in both axes. 

Fig. 2 is a general overview^ with detailed description of steps» of the process vf 
5 embedding an "imperceptible" identification signal onto another signal, 

E^. 3 is a step-wise description of how a suspected copy of an original is identified. 
Fig. 4 is a schematic view of an apparatus forpre-exposing Ohn with identification 
information. 

Fig. 5 is a diagram of a "black box" embodhnent. 
10 Fig. 6 is a schematic block diagram of the embodiment of Fig. S. 

Hg. 7 shows a variant of the Fig. 6 embodiment adapted to encode successive sets 
of input data with different code words but with the same noise data. 

Fig. 8 shows a variant of the Fig. 6 embodiment adapted to encode each frame of a 
videotaped production with a unique code number. 
15 Rgs. 9A-9C are representations of an industry standard noise second. 

Hg. 10 shows an integrated circuit used in detecting standard noise codes. 
Hg. 1 1 shows a process flow for detecting a standard noise code that can be used in 
the Fig. 10 embodiment 

Fig. 12 is an embodiment employing a plurality of detectors. 
20 Fig. 13 shows an embodiment in whkh a pseudo-random noise frame is generated 

from an image. 

Fig. 14 illustrates how statistics of a signal can be used in md of decoding. 
Hg. 15 shows how a signature signal can be preprocessed to increase its robustness 
in view of anticipated distortion, e.g. MPEG. 
25 Kgs. 16 and 17 show embodiments in which information about a file is detailed 

both in a header, and in the file itself. 

Figs. I S*20 show details of embodimenu using rotationally symmetiic patterns. 
Fig. 21 shows encoding "bumps" latiicr than pixels. 
Figs. 22-26 detail aspects of a security card. 
30 Fig. 27 is a diagram illustrating a network linking method using information 

embedded in data objects that have inheimt noise. 
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Figs. 27A and 27B show a typical web page, and a step in its encapsulation into a 
self cxtracring web page objea. 

Pig. 28 is a diagram of a photographic identification docum^t or security card. 

Figs. 29 and 30 illustrate two embodiments by which subliminal digital graticules 
5 can be realized. 

Fig. 29A shows a variation on the Fig. 29 embodiment 

Figs. 31 A and 3 IB show the phase of spatial freqaoicics along two inclined axes. 

Figs. 32A - 32C show the phase of spatial frequencies along first, second and third 
concentric rings. 

10 Figs 33A - 33E show steps b the registratirai process for a subliminal graticule 

using inclined axes. 

Figs. 34A - 34E show steps in the registration |Ht)cess for a subliminal graticule 
using concentric rings. 

Figs. 35A - 3SC shows further steps in the registration process fcx- a subliminal 
1 5 graticule using bclined axes. 

Figs. 36A - 36D show another registration process that does not require a 2D FFT. 
Fig. 37 is a flow chart summarizing a registration process foF subliminal graticules. 
Fig. 38 is a block diagram showing principal components of an exemplary wireless 
telephony system. 

20 Fig. 39 is a block diagram of an exemplary steganogrx^hic encoder that can be used 

in the telephone of the Fig. 38 systent 

Fig. 40 is a block diagram of an exemplary steganogra^hic decoder that can be used 
in the cell site of the Fig. I system. 

Figs. 41 A and 41B show exenqiiary bit cells used in one form of encoding. 
25 Fig.42 shows a hierarchical arrangement of signature blocks* sub-blocks* and bit 

cells used in one embodiment 

Hg. 43 shows a general overview of a computer system linked by using 
information embedded in the objects. 

Figs. 44-59 are representative corqputer screens generated in accordance with 
30 the present invention. 
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Fig. 60 is a basic ovoview of how a traditional file header can be rq>Iac6d by 
a pointo' to a database location where header inforniati(»i resides. 

Fig. 61 expands on Fig. 60 by illustrating that a pointer to a remote location 
can coexist with existing data file formats which might include headers. 
5 Fig. 62 is a basic map showing how an instrument or s^plication can associate 

a pointer attadied to an object to some remote digiUt] storage location. 

Fig. 63 is a condensed depiction of a process that an instrument or ^plicadon 
might go through in resolving and acting upon remote information pointed to by an 
attached pointer. 

10 Fig. 64 graphically illustrates that attached information can actually be 

sequences of actions or multiple sets of ntformation. 

Fig. 65 shows how the basic principles of master identity objects can be used 
to fiael a reporting iKtwork wherein a central location can "keep track" of where its 
copies migratie. 
15 Detailed Description 

In the fo]k>wing discussion of an illustrative embodiment, the words "signal" 
and "image" arc used interchangeably to refer to both one, two, and even beyond two 
dimensicRis of digital signal. Examples will routinely switch back and forth between 
a one dimensional audio-type digital signal and a two dimensional image-type 
20 digital signal. 

In arisT to fully describe the details of an illustrative embodiment, it is 
necessary first to desaibe the basic properties of a digital signal. Fig. 1 shows a 
classic reprcsenution of a one dimensional digital signal. The x-axis defines the 
■ index numbers of sequence of digital "samples," and the y-axis is the instantaneous 
25 value of the signal at that sample, being consu-ained to exist only at a finite number 
of levels defined as the "binary depth" of a digital sample. The example depicted in 
Fig. 1 has the value of 2 to the fourth power, or "4 bits," giving 16 allowed states of 
the sample vahie. 
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For audio iofonnadcm sncfa as sound waves, it is commonly accepted ihat the digicizatiooEi process 
discrerizw a CTninftntw pbeoomeca both in ibe time domaia and in the sigoal levd domain* As such, the 
process of digitizatioA iiseJf mtxoduces a fusdamemai erm source, in that it cannot itcord deudl smaller 
than die discretization intervaJ in either domatn. The iodnstiy has refniEd to this, among odser ways, as 

5 "aliasing" in d!e time domain, and ''quaniizaiion noise" in ciie signal level domain. ThuSi tbers will always 
be a basic error floor of a digital signal. Pure quaotizatioQ noise, measured b a root mean square sense. 
Is thcorcticaiiy known to have the value of one over the square root of twelve^ or abom 0,29 DN, where 
DN stands for 'Digital Number' or the finest unit increment of the signal ievel. For example, a per&a 
12-bit will have 4096 allowed DN widx an hmate looc mean square noise floor of -0.29 DH. 

10 AH known pi^sical measurement processes add additional noise to die transformadon of a 

cominnous signal into die digital foim. The quaniizaiion noise typically adds in quadramte (square root of 
die mean scpiaxes) to the 'analog noise" of die measuiemenc process, as it is sometimes tefened to. 

ahnosc all commercial and technical processes, the use of die decibel scale is used as a 
measure of signal and noise in a given recording medium. Tlie expression *signal-to-noise ratio* is 

15 generaUy used^ as h wiU be in diis disdosure. As an example, this disclosure rrfecs to signal to noise 
ratios in terms of stgsial power and noise power, dius 20 dB represents a 10 tinns increase in signal 
amplitude. 

In smnmaiyp diis embodiment embeds an N-bic value onto an cnnie signal through die addition of 
a very low »tnp«mHft encodatian signal which has tide look of pure noise. N is usually ai least 8 and is 

20 capped on the higher end by ultimaie $igna]-co-ooise consideradons and *bit error" hi retrieving and 

A ^r}tV^t^^ die N<4)it vahie. As a practical maner, K is chosen based on ^iicadon specific consideratxcms, 
qirh as ±e number of unique different "signanares" dtat arc desired. To illustrate, if N = 123. then die 
number of unique digital signanircs is in excess of lO'^'^S (2^123). This number is believBd to be mare 
than adequate to bodx identiiy die material widi sufficient statistical certainty mid to index exact sale aid 

25 distribution aif^^m^aiton. 

The ampHnide or power of diis added signal is determined by die aesthetic and uafoxmatianal 
cQoslderanons of each and every application using die present mediodology. For instance, 
oon-pro&ssional video can stand to have a higher embedded sigtial level wtdiout becommg noticeable to 
die average hmnair eye, while high precision audio may only be able to accept a rdativeiy small signal 

30 level lest the human ear peroexvc an objecdonable increase m Inss.' These statements are gcneralides and 
each applicadon has its own set of criteria in choosing die sigoal level of die embedded id e nrificni on 
signal. The higher the level of embedded signal, die more corrupted a copy can be and still be identified. 
On die odser hand, die hi^ die level of embedded signal, the more objectionable d« perceived noise 
might be, potentially impacting the value of the distributed material. 
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To illusciate the range of different applicatum u> which applicant's technology can be applied, the 
piescni qjecificaricn details two diffcrem systems. The first (tenned, for lack of a better nairw a 'haich 
encoding* system), applies identiflcatioa coding to an existing data signal. The second (termed, lack of 
a better name, a 'real time encoding" system), applies identification coding to a signal as it is produced. 
5 Those skilled in die an will recognize that the princtpies of applicant's technology can be applied in a 
number of odier contesos in addition to these particularly described. 

The discussions of diese two systems can be read in either order. Some readers may find die 
lazier moze tnnutive than die former; for others the cootraiy may be true* 

10 PATCH ENCOPff^Q 

The Mowing discussion of a fizsi class of embodiments is best prefaced by a scctioa defining 
zeievanc tenns. 

The originai signal refers lo either die original digital signal or the high (pialiiy djgifjyf^ copy of a 
non>4igital original. 

15 The N-bit identification word refers to a unique identification binary vaiue^ typically having N 

range anywhere from A to 128. wiiidi is die identification code ultimately placed onto the original signal 
via the disclosed transformarion process. In die illustrated embodiment, each N-bit identification word 
begins with the sequence of vahies *0101/ whidi is used to detemsne an optimizaxian of the signaL-to- 
noise ratio in the identification procedure of a suspen signal (see de&inon below). 

20 The m'th bit value of die N-bit identificanoa word is either a zero or one corxespoading to die 

value of the m'th place, reading ieffc to righc, of the N-bit word. E.g., the first (za<»I) bit value of the 
N»ft identificadon word OlllOlOO is die value '0;' the second bit value of diis identification word is *i\ 
etc. 

The m'th mdividual embedded code signal refers to a signal which has dimensions and cjctent 
25 predsely equal to die originai signal <e.g. both are a 512 by S12 digital image)» and which is (m the 

illustrated embodiment) an independent pscado*randQm sequencie of digital vahxes. *Pseodo* pays homage 
to the difficulty in phflosnphirafly defining pure randomness* and also indicates that there are various 
acceptable ways of generating the 'random** signal. Thcxc will be exactly N individual embedded code 
signals assodand widi any given oiigioal signal. 
30 The aceeiwable perceived noise level refers to an appiicatioii-specific decemdnation of how mncfa 

''extra noise/ i.e. amplitude of the composite embffdded code signal described next, can be added to die 
original signal and still have an acceptable signal to sell or odicrwise distribute. This disclosure uses a 1 
dB increase in noise as a typical value which might be acceptable, but dus is quite arbitrary. 

Tlie comtwsite embedded code signal refers to the signal wiiidi has diinfflslons and extent 
35 precisely eipial to the original signal, (e.g. bodi are a 512 by 5 12 digital image)* and which contains die 
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addition and appropriate atienuaitoa of tfae N mdividoaS embedded code signals. The individual embedded 
signals ate generaed on an artritmy scale, wfaezeas the ampiimde of ibe composite sigaal nnist not exceed 
the pre-set acceptable perceived noise level, besce tbe need for "attenuation'* of the N added individual 
code signals. 

5 Tbe ^wtribntaMe ^ienal refers to the nearly similar copy of tbt original signal, oms ist tng of ifae 

otigina! signal plus dte composite embedded cocte sipal. Tbis is the signal which ts disthboied (o die 
outside comnmnity, having only digbtly higher but acceptaJiIc ^noise pftqmies'* than ibe originai. 

A CTis pgct stf flai refers to a signal wbidi has tbe general appearance of die origiBai and distzxteaed 
signal and whose potsoiial identificatioa match to the origitsal Is being questioned. The suspect signal is 

10 then an^yzed to see if it matches the N-bu identi&cation word. 

The detailed methodology of this first embodimenic bcgiiu by stating that tbe N-bic identification 
wont is encoded onto the original signal by having each of die m bit vahies anUti[riy tbdr corresponding 
individual embedded code signals, the resultant bemg accnmulared in the composite signal, die fuUy 
summed composite signal then being ^nt^nn^n^ down to the acceptable perceived noise amplitude, and the 

15 resultsnc composiiB signal added to the origioal to become tbe dssixibutable signal. 

The originai signal, die N*blt Identification word, and all N hxlividnal erobedded code signals are 
then stored away in a secured place* A suspect signal i$ dien found. This signal may have undergone 
oEuldplc copies* comprcssians and deconqiressioQs. resamplings onto different spaced digital signals, 
transfers tsom digital to analog back to digital media* or any conthtnarion of these items. £ the signal still 

20 appears similar to die oiigiaai, i.e. its innate quality is not Uioroughly destroyed by all of diese 

transfonnations and noise additions; tbea depeodmg on the ^gnai to noise propodes of tbe e m bedded 
signal, die ideniificadon process should foacdon to some objectiYC degree of statisdcal a)nSdence. The 
eiaent of amupri on of the suspect sxgtial and the original acceptable perceived noise levd axe two key 
paxameteis in detemdiung an expected confidence level of id^fication. 

25 The idoitification process on die suspected signal begins by lesampling and aligning the suspected 

signal onto the ^gxtal format and e«ent of die originai signal. Thus, if an image has been reduced by a 
faaor of two, it needs to be digitally enlarged by that same fecior. Likewise, if a piece of music has been 
'cut out,' but may still have die same sampling rate as the original, it is necessary to register this cus-oot 
piece to the ohgtnal typically done by performing a local digital cross-comlation of die two signals (a 

30 common digital operadoa), findmg at what delay value die ccjnelatian peaks, dien osmg dns found delay 
vsdoe to register die cut piece to a segmeor of die original. 

Once die suspect signal has been sams^e-spaemg matched and registexed to the original, die signal 
levels of die suspect signal should be matched in an rms sense to die signal level of die origmal. This can 
be done via a search on die paramefers of of&et, amplification, and gamma being optiioized by using the 
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m^TiifTifim of the mean squared error between die cwo signals as a fuDcdon of the ctoe parameters. We 
can call ttie suspect signal aormalized and regiscmd at diis point, or just soimaiized for convenience. 

Tbe newly matched pair then has die original signal subtracted fium the aorstalized suspect ngnal 
to produce a difoem signal. The diflerentx signal is then ctosfr-coizeiated with each of die N isdivuhial 
5 embedded cc<ie signals and the peak cross^rreiatioa vahie recorded. The fits four bit code (*OIOr) is 
used as a calibrator both on the mean values of ±t zero value and \tc one vahie, and on ttaher 
registration of die two signals if a finer signal id noise rado is desired (Le., the opdmai separ^on of the 
0101 signal will indicate an optimal registration of the two signals and will also indicate the probable 
existence of the M-bit iftantificatinn signal bemg present) 

IQ The icsulting peak cross-correiatian values will form a noisy series of floating point numbers 

which can be transfoimed into O's and Vs by their proxxmicy to die mean vahies of 0 and 1 found by the 
0101 caiibranon sequence. If the suspect signal has indeed been derived from die original, the 
idemification number resulting bom die above prtxxss wiU match the N-bii td e ndficarion word of die 
nrigifwi, bearing In mmd eifher predicted or nnknown "bit error* statistics. Signal-tOHioisa consideraiioos 

IS will determine if there will be some kind of *bit eniTr" in die idendfication proce^ leading to a form of 
X% fffobability of idcntificalioQ where X mi^ be desired to be 99.9% or wliatcvcr. If the suspect copy 
is indeed not a copy of ttic original an cssendaily random sequence of O's and Vs will be produced, as 
weU as an apparent lack of separation of die resultant vahies. This is to say, if the resultant vahies are 
plotted on a histogranu die existence of die N-Wt identrficarion sigiml wHl exhibit strong bi-level 

20 diaracterisdcs, whereas die non-existence of the code, or die existence of a different code of a different 
original, wiU exhibit a type of random gaussiaii<4ike distribution. This histogram separation alone should 
be arfRrw^r for an ideotificacion, but it is even stronger proof of idemification when an exact binary 
sequence can be objecdvely rq^roduced. 

25 Soecific Example 

^wiftgine diat we have taken a valuable picture of two heads of state at a cocktail pariy, pictures 
which arc sure to earn some reasonable fee in die commocial market. We desire to sell diis picmre and 
eosuxediat it is aot used ni an unauthorized or uncompensated manner. This and die following steps are 
summarized m Ftg. 2. 

30 Assmne the ptcmre is transformed into a positive color print. We &si scan diis into a digidzed 

torn via a normal high quaK^ black and white scamcr widi a typical [dxooometric spectral response cam. 
(It is possible to get better uldmate signal to noise ratios by scanning m each of the diree primary colors of 
die color image, but this anance is not central to describing the basic process.) 
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Let ti5 assume tbat the seamed image now becomes a 4000 by 4000 pixel monodirome digital 
image wiiii a grey scale accuracy de&xed by 12-bit grey values or 4096 allowed levels. We will call this 
the "origin^ {£gital image* realizing diat tiiis is the same as our 'original signal" in the ^ove definiiions. 

Daring ihe scanning process we bave arfaitrarxly set absolute black to correspond to digital value 
'30\ We estimate diat tbexe is a basic 2 Digital Number root mean square noise ousting on ±e original 
digital image, plus a dteotetical aoise (known in the industry as *sJtoc noise") of the square root of die 
brightness value of any given pixel. In fbnnuia, we have: 

<RMS Noi3e^> - sqrt<4 + {V^-30)) (1) 



Here^ n and m aie simple imiexing values on rows and columns of the image ranging from 0 to 3999. 
Sqrc is the square root. V is the DN of a given indexed pixel on die original digital image. The < > 
biackets around die RMS noise merely mdicates diat this is an expected average value* where it is dear 
that each and every pixel will have a random error individually. Thus, foe a pixel value having 1200 as a 
15 digital munber or "brighmess value*, we fmd diat Its expected rms noise value is sqrt(1204] » 34.70, 
whidi b quite close to 34.64, the square root of 1200. 

We ^mhcnnore realize dial the square root of the innate brightness value of a pixel is qoc 
precisely what die eye perceives as a mmtrnnTn objeaiamible noise, thus we come up with the formula: 

20 <RMS Addable Noisci,„> » X*sqrt(4+(V«,„-30rY) 0) 

Where X and Y have been added as empirical parameters which we will adjust, and "addable* noise reiiers 
to our acceptable perceived noise level from the defmitioas above. We now intend to experiment Viith 
what exaa vdne of X and Y we can choose* but we will do so at the same time that we are permming 

25 die next st^ in the process. 

The nea step in our process is to choose N of our N-bit identification word. We decide that a 16 
bit main idcmificadm value with its 65536 possible values will be suffidcmly large to idemify the iin^e 
as oms, and drat we will be directly selling no more dm 128 copies of die image which we wish to ttack, 
giving 7 bits plus an exghdi bit for an odd/even adding of the first 7 bits <i.e. an entsr chrdring bit on the 

30 firstseven). The total bits required now are at 4 bits to dw 0101 calSstation seq]^ 

ideotiiication. 8 Ibr die vetstoo, and we now dirow in aoodier 4 as 2 fimher error c he c lring value on die 
first 28 bits, ghnng 32 bits as H. The final 4 Uts can use one of many industry standard error checking 
methods to dioose its four values. 

We now randomly dg rrr^"^ the 16 bit m^^n klentifxation number, finrlmg for example, 1101 

35 0001 1001 1 1 LO; our first versions of tiie original sold will have ail O's as die version identifier, and the 
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enor <*h ^'^g tnts will &il out where tbey toay. We now have our onique 32 bit ideotificatioii wocd 
wtaidi we will embed on &e origmal digital im^t. 

To do this, we geoenue 32 indepeodem candom 4000 by 4000 eocoding images for each bit of oui 
32 bit identificadoa word. The maimer of generacmg duse random images is revealing. Tbere ate 
5 numerous ways generate these. By far the simpiest is to turn up the gain on the same scanner diat was 
used to scan in the original photogra^^, only this time placing a pure blade image as the input, then 
yrarming this 32 times. The only drawback to this technique is that it does lequire a large amount of 
memory and that *&ted pattern** noise win be pan of each independent 'noise image." But, the fixed 
paoem noise can be removed via nonsal 'dark 6ame'* subtraction tedmiques. Assume diai we set the 
10 absolute black average value at digital number ' 100/ and that rather tiian findmg a 2 DN rms ooise as we 
did in the normal gain settings we now find an ims noise of 10 DN about each and every pixel's mean 
valne. 

We next apply a mid-spacial-teiuency baiKipass filter (spaiiai convointion) to each and every 
independent random image, essemiaUy removing the very high and the very low spatial fieqtiencies from 

15 tbim. We icmove the very low frequencies because simple real-world enor sources like gccHnctiicai 
waipii^, splotches on scacneis, mis-registradons, and die Like will exhibit themselves most at (ower 
frequencies also, and so we want to concentrate our identification signal at higher spatial frequencies in 
order to avoid these types of corruptions. Likewise, we remove die higher frequencies because multiple 
generation copies of a given image, as well as compressicm-decompicssion transfDzmations, tcod to wipe 

20 out higher frequeodcs anyway, so there is no point in placing too much tdeniification signal into these 
frequencies if they wiU be the ones most prone to being anenoated. Therefbxe, our new filtered 
independent noise images will be dominaxed by mid-spatial frequencies. On a practical notc» since we are 
using 12-bit values on our and we have removed the DC value effectively and our new rms noise 

will be slightly less dian 10 digital numbers^ it is useftil to boil this 4own to a 6«bit value ranging from -32 

25 through 0 to 31 as the rsultant random image. 

Next we add all of the random images together which have a T in dicir corresponding bit value 
of the 32-bii idencficaikm word, yxutnuiaring the result in a 16-bit signed integer image. This is the 
iinAtrm'^*^ and un-scaied venion of the composite embedded signal. 

Next we ^eriment visually with adding the composite embedded signal to d» original digital 

30 image, dtcough varyhig the X and Y pazameten of equarioa 2, In formula, we visually iterate to bodi 
ft^i^ miTi! X and to find the appropriate Y m the following: 

= + V^.„*X*sqrt(4+V^.*Y) (3) 
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where disi refers to ihe candidate distributable inuge, i.e. wc arc visually iterating to find what X and Y 
will give as a& acceptable image; orig refers to the pixel vahje of the origioal image; and comp refers lo 
the pixel value of the composite image. The n's and m's still index rows and coiumns of die image and 
bdicate thai this operation is done on all 4000 by 4QQ0 pixels. The symbol V is die DK of a given pixel 
5 and a given image. 

As an arbitrary assumption, mw, we assume diat our visual cqiexinicntadon has found that die 
value of X= 0.025 and Y^O.6 are aaaptabie values when annparing the original image with the 
candidate distributable image. This is to say, die distributable image widi the 'extra noise" is acceptably 
dose to die original in an aesdietic sense. Note diat since our individual random images had a random rms 
10 Qoise value aroimd 10 DN, and that adding appiDximaiely 16 of tiiese images together will inirrease che 
composite noise to around 40 DN, die X mukiplication value of 0.025 will bring die added rms noise bade 
to around 1 DN, or half die amplitude of our umace noise on die origiuBl. This is tooghiy a I dB gain in 
noise at die dark pixel values and oonespondingly more at die brighter values modified by die Y value of 
0.6. 

15 So widi diese two values of X and Y, we now have constructed our first versions of a 

distributable copy of die originaL Other versions will merely create a new composite signal and possibly 
change the X siighdy if deemed necessary. We now lock up die original digital image along widi the 
32-bii identificaijon word for each version, and die 32 mdcpendem random 4-bii onages, waitmg for our 
first case of a suspected piracy of our original. Storage wise, diis is about 14 Megabytes for die original 

20 image aod 32*0.5byics*i6 million =« -256 Megabytes for die ranttom mdividuai encoded images. TTiis is 
quite a^^cp^^^*^ for a single valuable image. Some storage economy can be gained by simple lossless 
compression. 

flyf^o :> a«n«t«l Piracy of our Image 
25 We seU our image and several motshs brer find our two he2ds of state in the exact poses we sold 

diem in, secnringly cut and lifted out of our image and placed into another styli2ed background scene. 

Tliis new "suspect" image is being printed in 100,000 copies of a given magazine issue, lei us say. We 

now go about deicrminmg if a portion of our original image has mdeed been used in an unauthorized 

masnsr. Fig. 3 y^^^nrr'"^^* the details. 
30 Thefirststq^istotakean issue of Che magazine, cut out die page wiUi die image on it, dien 

carefully but not too carefirily cue out die two figures from die background hnage using ordinary scissors. 

If possible, we will cm out only one couDccted piece radier dian die two figures separately. We paste diis 

onto a black background and scan this hxio a digital fotSL Next we electronically flag or mask out die 

black background, which is easy to do by visual iospectioa. 
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We now px€cuie die oiigiaal digital imag« firom our secured place aloi^ widi the 32-bic 
identiRcadoa woid and die 32 individual embedded images. We place die original digital image onto our 
computer screen using staDdaid image manqmlation software, and we rougbiy cut along die same borders 
u our masked area of die suspect image, masking diis image at the same cimc in rough!/ the same 
5 manner. Tlie word 'roughly' is used smce an exact oining is sot needed, it merely aids die uientificauon 
statisdcs co get it leasanabiy dose. 

Next we rcscaie ±c masked suspea image to roughly match the size of our masked original digital 
image, that is» we digitally scale up or down die snspea image and roughly overlay it on die original 
image. Once wc have performed diis rough registration, we diea dirow die cwo images into an ffl i T o m a reri 

10 scaling and registiaiian program. The program performs a search on die three parameters of x posiuon, y 
posmon, and spatial scale» with die figme of tnerls bdng die mean squared error between d» two Images 
given any given scale variable and x and y offset. Tbis is a faiziy standard nnage processing methodology. 
Typically this wonld be done using generally smooth interpolation techniques and done to sob-pixel 
accQxacy. Ibe search method can be one of many, where die simplex method Is a typica] one. 

15 Once die optimal scahng and x-y position variables are found, next comes anodier search on 

oprimizing the black level, brighmcss gain, and gamma of the two imagw. Again, die figmc of merit to 
be used is mean squared error, and again the simplex or other scaitA methodolcgies can be used to 
optimize die diree variables. After dicsc dirtc variables are optimized, we apply dieir correaiwis to die 
snspcct image and align it to exacdy die pixel spacing and masking of die original digital image and its 

20 ^^^^ We can now call this the standard mask. 

The next step is co subtract the original digital image from die newly ootinalized suspea image 
only widnn the standard mask region. This new imstge is called the difference image. 

Then wc step diioi# all 32 individual random cnd)cdded images, doing a local cross-correlation 
between the masksd difference image and the masked individnai embedded image, 'Local' refers to die 

25 idea dutt one need only start correlating over an offset region of +/- 1 pixels of offeei between the nominal 
nsgistrarionpointsof ±cr«ro images found durmg die search procedures a^^ Tbe peak correlation 
should be very dose to the nominal registradon point of 0,0 offect, and wc can add die 3 by 3 correlation 
vahies togedier to give one grand correlation value for each of die 32 individual bits of our 32-bii 
identification wgrL 

30 After doing diis for ail 32 bit places and dicir corresponding random images, we have a 

quasi-floating point sequence of 32 values. The first four values rcpresem our calibration signal of 0101. 
Wc now take die mean of die first and diixd floating poim value and ^ vahic and 

we take die mean of die second and die fourth value and caU diis floari^^ 'l-* Wedienstcp 

through an remaning 2S bit values and assign ddicr a *0' or a T based simply on which mean value diey 

35 arc doscr to, dated simply, if the suspect image is indeed a copy of our original, die embedded 32-bii 
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resuiting code should match that of our records, and if it is not a copy, we should get general randoixmess. 
Hie third and the founh possibilities of 3) Is a copy but doesn't match i den tifi ca ti on sumher and 4) bn't a 
copy biK does matdi air« in the case of 3)» possible if the signal EO noise laiio of the process has 
pluimnetcd, i.e. tbe •suspect inage' is tnily a very poor copy of the origiiial, and in die case of 4) is 
basically one f h «'>«^ in four billion since we were using a 32-bic identification munber. If we are tniiy 
wQiiied about 4}» we can just have a second uidcpendcnt lab perform their own tests on a different issue of 
die same maganne, Finafly, chedang the enor-checlc bits against what die values give is one final and 
possibly ovctldll check on the whole process. In siniati^ where signal to noise is a passible problem, 
these ernir rhfcikrn^ bits mi^ be eiixninaicd without too mnch harm. 



Benefits 

Now diat a foil dcscriptioo of the first embodiment has been described via a detailed example, it is 
appropriate to poim out the rationale of some of ihe process steps and their benefits. 

The ttitimfg benefits of the foregoing process are that (ri>taining an idendfic^tioa number is fully 
mdependcnt of d» manncB and methods of preparing fte difference usage. That is to say, the manners of 
preparing die difference image, such as cutting, registering, scaling, eitxtcra, cannot increase dac odds of 
finding an idemificaiion niimher when none exisis: it only helps die agnal-tD-ooisc ratio of the 
identificaiion process when a true identification number is present. Methods of preparing images for 
idcmificarion can be difecnt from each other even, providing the possibility for mnltiple mdepeodcni 
20 mcdiodologics for maidng a match . 

The ability to obtam a match even on sub-sets of dse original signal or image is a key point in 
today's mfoimaiioa-rich worfd. Curdng and pastix^ both hnages and sound clips is becoming more 
common, allowing such an embodiment to be used in detecting a copy even when origmal material has 
been thus connptcd. Rnaljy, the signal to noise ratio of matrhmg should begin to become difficult only 
25 when tbe copy marcriai itself has been significantly altered either by noise or by significant distortion; both 
of these also will affect dial ce^^'s commerdai value, so chat trying to thwart the system can only be done 
at the expense of a huge decrease in commercial value. 

An early amcepdon of this technology was the case where only a single "snowy image" or 
random signal was'added to an original image, i.e. the case where N=l. "Decoding" this signal would 
30 involve a subseqocat ipatitiWMttcal analysts using (generally statistical) algoritinas to make a judgment on 
the pieaaice or ^aoce of this signal. Hie reason this approach was abandoned as the preferred 
fffl ho^jfupfif was that diere was an inherent gray area m Urn certaonty of detecxing die pieseoce or absence 
ofdiesignaL By moving onward to a muliinide of bU planes, i.e. N > l, combined with snnplc pre- 
defined algoridmB prescribing die manner of choosing between a t)- and a the cenainiy qucsncm 
35 moved from the realm of cxpcn statistical analysis into die realm of guessuig a random binary event such 
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as a ooin flip. This is seen as a powerful feanne teiative to (he innxitive accepcaiice of this technology in 
bodi the oounroom and the markecpiace. The analogy which snmmarizes the mvemor's (houghis on this 
whole question is as follows: The search for a single identificaiion signal amoams to calling a coin flip 
only once, and refybig on arcane sxpens to make the call; whereas the N> 1 embodiixKnt relies on the 
5 broadly inmitive principle of coneccly calling a coin flip N times in a row. This simatioa is greatly 
exacerbated, Le. the problems of "interprecafioo" of the presence of a single signal, when images and 
sound dips get mniw and smaller in extent. 

Anoifaec inxponam reason that the N> 1 case is preferred over the N=I emhodiment is that in the 
N^I case, the manner in which a suspect image is prepared and manipaiaied has a direct bearing on the 

10 likelihood of maicing a positive identification. Thus, the manner with which an expert makes an 

identification descrmination becomes an hu^ral pan of that determination. The existence of a mnltimde of 
mathematical and statistical approaches to makmg this determination leave open die possibility that some 
tests might maice positive identifications while others might make negative determinations, inviting funfaer 
arcane debate about the relative merits of the various idemificacion ap]»Q3Ches. The N > I embo<Unient 

15 avoids this fhrdier gray area by presenting a m^od where no amotmt of pre-processing of a sigxud - other 
than pre-processing which suircpndoasly uses knowledge of the private code signals • can increase ±e 
likelihood of •calling die coin flip N times in a row.' 

The fullest expression of the present system will come when it becomes an industry standard and 
nnmerons ind^mdeot groups set op with their own means or 'ui-honse' brand of applying end^edded 

20 identificaiion mmibets and in their decipherment. Ntimerous independent group kientification will fimher 
ffl ha p rt> [lie ultimate objectivtty of the method, thereby enhandng its appeal as an mdustry standard. 

Use of True Polarity in Creatine ±e Conmoaite Embedded Code Signal 

The (brcgoiDg discussion "laHg use of the 0 and I formalism of binary technology to accomplish 
25 its ends. Specifically, the O's and Ts of the N-bit identification word directly amhiplied their 

corresponding mdividuai embedded code signal to fotm die composite embedded code signal (step 8» Fig. 

2). This apprxiacfa certainly has its conoepniai simplicity, but ihc multiplication of an embedded code 

signal by 0 along widi the storage of that embedded eodz contaios a kind of mefficieocy. 

It is prefored to maimain die fonnatism of the 0 and 1 nature of the N-bic identificaiion word, but 
30 to have du O's of die word induce a sufatraaion of disir cotxcsponding embedded code signal. Thas. in 

step 8 of Rg. 2, rather than only 'adding' the individual embedded code signals which correspond to a *!' 

in the N-bit identification word, we will also 'subttact' the individual embedded code s^nals which 

correspond to a '0' in the N4)it identification word. 
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At first glance this seems co add more apparent noise to (be fmai composite signal. But it also 
increases the energy«wise separadoa of the 0*s Erom the Vy and thus die 'gain' wliich is applied in stq> 
10, Fig. 2 can be concspondrngly lower. 

We can refer to diis imptovemeni as the use of true polarity. The main advantage of this 
5 improvement can lately he sHmmnriyrrf as 'infozxnationai efficiency/ 

'Perceonial Orthogonality' of the Individual Embedded Co rf^ Sitrnai^ 

The fbitgotng discussion contemplates the tise of genezally random ooise-likc signals as the 
individual embedded code signab. This is pcihaps die simplest form of signal lo gcnerare. However, 

10 there Is a form of mfbrmadonal oprimizarioa which can be applied to the set of the individual embedded 
signals, which the applicant describes ond^ the rubric 'perceptual orthogonality.* This term is loosely 
based on the mshematiGal concept of die orthogonality of vectors^ widi die current additional requtrcmenc 
thai diis onfaogonaiity should maifmiTB die signal energy of the idennficadon inSoixnation while 
mpifltaimng it bciow some perceptibility threshold. Pot another way, the embedded code signals need not 

15 necessarily be random in nanse. 

Trfct> jmd ImpTDvements of die Ftr ^ gmhndfmgnf in the Pidd of Emuisioii^Bascd PhotographY 

The foregoing discussion outlined techniques dial are applicable to photographic materials. The 
following section explores the details of this area ftirdser and discloses ceitam improvements which lend 

20 diemsetves to a broad range of {^plications. 

The first area to be discussed involves the pre-appiication or pre-exposing of a serial number onto 
tradidooal photographic products, sodx as negative film, print paper, transpaiencies* etc. In general, this 
is a ws^ to embed c priori unique serial numbers (and by implication, ownership and oraeking infotmatioa} 
into photographic material. The serial numbers diemselves would be a permanent part of die noimaily 

25 oqposed picture, as opposed 03 being relegated to die margins or stamped on die back of a printed 

photograph, which all require sq^arare locations and sqjaraie mediods of copyhig. The ^serial nmnber* as 
it is caDed here b generally synonymous widi the N-bit identification word, only now we arc using a more 
^ ' lOmi f W ' industrial terminotogy. 

hi Fig. 2, step 1 1, die disclosure calls for the storage of die 'original [image]* akmg widi code 

30 images. Then In Fig. 3, step 9, it directs diat the original be subtracted from the suspect image, dieieby 
lesviAg die possible idemificatioa codes plus whatever noise and corruption has accsmuiated. Therefore, 
the previous disclosure made die tacxi assumption diat diere ousts an original widymt die composite 
embedded signals. 
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Now is the case of seiliiig print paper aad oihst duptkatioa film piodscxs, this will still be the 
case, i.e., an 'ongtnai" without the embedded codes will indeed exist and the basic methodology of die 
fint embodiineni can be employed. The original Glai serves perfectly well as an "ucencoded original/ 

However, in the case where pre-exposed negative iilm is used* the composite em^dded signal 
5 pre-exists on d» original &lm and duis there will never be an **origmal* separate from the pre-embedded 
signal. It is this Latter case, ihezefbie, which wiD be examined a bit more closely, along with observations 
on bow to best use the principles discussed stove (the former cases adhering to the prcvtousiy outlined 
methods). 

The deaiesi point of depamire for the case of pre-nmnbered negactve film, i.e. negative film 

10 whidi has had each and evesy firame pre-esq^sed wich a very &int and unique composite embedded signal, 
comes at stq) 9 of Fig. 3 as previously noted. There are cextainiy odier differences as wdl, but they are 
mostly logistical in nacuie. such as bow ami when to embed the signals on the film, how to store the code 
numbers and serial numb^, eo:. Obviously the ptenexposing of fiha would involve a m^'or change to the 
general producdon process of crcati&g and packaging film. 

£5 Pig, 4 has a schematic outlining one potenrial post-hoc mechanism for pre-exposing fihn. Tost- 

hoc' refers to flying a process after the full common manufacmring process of film has already taken 
place. Evcnmally, economies of scale may dictate placing this pre-exposing process directly into the chain 
of mamifacmring film. Depicted m Fig. 4 is what is commonly known as a film writing system. The 
computer, 106, displays the composite signal produced m si^ 8, Fig. 2, on its pho^or screen. A given 

20 frame of film is diea exposed by imaging this phosphor screen, wbm the exposure level is generally very 
faint, ix. generally impeiceptifaie. Qeariy. the marketplace will set its own demands on how faint this 
should be, that is, the level of added 'graininess' as practitianers would put it. Each frame of film is 
seqnentiaily exposed, where in general the composite image displayed on the CRT 102 is changed for each 
and every frame, thereby giving each trame of film a different serial number. The transfer lens 104 

22 highlights the focal amjogate planes of a film frame and the CRT face. 

Getting back to die applying the principles of the foregoing embcdimHU in the case of pre-exposed 
negative film... At st^ 9, Fig. 3, if we were to subtract the 'original" with its embedded code, we would 
obviously be "erasing" the code as well since the code b an integral pan of the origmal. Fortunately, 
remedies do exist and ideotificaiions can still be made. However, It will be a cha l lrage to artisans who 

30 refiw diis embodiment to have the signal co noise ratio of die identification process m the pre-exposed 
negative case qspiiiach die signal to noise ratio of dae case where die no-encoded origmal exists. 

A mrrinrr definition of the problem is m order at this poinL Given a sospea picture (signal), 
find the idemification code IF a code exists at al. The problem reduces to one of finding the 

amplimdc of each and every individual embedded code signal within the suspect picmre, not only within 

35 the comexi of noise and corruption as was previously e^lained, but now also within the context of the 
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coupling betveea a capnixed image and the codes. 'Coupling' here refers to ±e idea dliat the captuted 
image 'randomly biases" the cross-coireiation. 

So, bearing in mind ihis additiooal item of signal coupling. Uie identificatioo process now 
estimates the signal ampUcude of each and every tndividuai embed d ed code signal (as exposed to taking the 

5 cross-corrdanan resale of step 12* Fig. 3). If our identificazian signal exists in the suspect picture, the 
ajspUtudes thus found win spUt into a polarity with positive ampiimdes being assigned a *r and negative 
amplitudes being assigned a '0'. Our miqpe idamScsmn code manifests itself. If. on the other hand, no 
such identification code exists or it is someone else's code^ dm a csoidam gaussian-like distribution of 
amplitudes is found with a random hash of values. 

IQ it remains to provide a few more details on how die amplitudes of die individual embedded codes 

are found. Again, fommaiely, diis exact problem has been oeated in odier technologic^ applications. 
Besides, ihiow dixs problem and a Me food into a crowded room of mathemadcians and sutistidans and 
satxAy a half dozen opdmized methodologies will pop out after some reasmiabie period of time. It is a 
rather cleanly defined problem. 

15 ■ One specific example solution comes torn the field of astronomicai imasing. Here, it is a macutc 

prior an to subaaci om a "theimai noise ftame* from a given CCD image of an object. Often, however, 
ic is not precisely knowa wliat scaling factor to uso in subtracting the thermal &ame» and a search for the 
cocrea scaling factor is performed. Tliis is precisely the task of diis step of the presw embodiment. 
General practice merely performs a common search algorithm on the scaling factor, where a 

20 scaling faoor is chosen and a new hnage is cieaied according to: 

NEW IMAGE =» ACQUIRED IMAGE - SCALE * THERMAL IMAGE W 
The new image is applied to the &st fburier transform routine and a scale factor is eventually 
fbumi wfazdi n^F™'*^^ the intcgsned high frequency contczit of the new ImagE. Hiis gmend type of 
search opeiaiion with its mmimiiation of a particular quantity is exceedingly oommon. The scale factor 

25 tfans found is the sought-for *ampiitude/ Refinements wfaidi are contemplated but not yet inyi ememe d 
are where the coupling of die higher derivatives of die acquired image and the embedded codes are 
estimated and removed from the ralriTlaTfri scale factor. In other words, cenain bias effects from the 
coupling mendoned eartier are present and should be eventually accounted for and removed both dxrough 
theoretical and empirical experimeocaxion. 

30 

TTcy antt Tmpfavements In the Detection of Signal or Image Alteration 

Apan from the basic need of identifying a sigiud or image as a whole, there is also a rather 
ubiquitous seed to deteci possible alterations to a signal or image. The fallowing section describes how die 
foregoing emfaodimcflt, with cenain mociifications and improvements, can be used as a powerful tool in 
35 rtiis area« The potendal scenarios and applicaiiaiis of detecting aiteiatioDS are iimumezablei 
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To first summarize, assume (tut we have i given signal or image whidk has been positively 
idcmified using the basic methods oatiioed above. In odier words, we know Lis N-bti idemificaaon word, 
lis isdividuai embedded code signals, and iis composice embedded code. We can then faiiiy simply create 
a spatial map of dse composite code's amplitude witiiia our given signal or image. Funhcniiore, we can 
5 divide this amplitude map by die known composi^ code's spatial amplimde, giving a normalized map, i.e. 
a map which shooJd flocniate abcsut some global mean value. By simple examination of this map, we can 
visually detect any areas which have been significantly alceied wherein the value of the nonnalized 
amplitude dips below some stadstically sa thresinld ba^ purely on typical noise and ootmption (eiror). 

The details of implementing die creatioa of the amplitude map have a variety of choices. One b 
10 to perform die same proceduie which is used to determine the signal amplitude as described above, only 
now we step and repeat die nmltiplodon of any given area of the signal/image with a gaussian weight 
flmciioa centered about die area we are investtgatiflg. 



^^nr'TT'l ^'^"^ ^"^"^ ^^^^ 
15 7^ disclosure dius (ar has outlined how each and eveiy souzce signal has its own unique set of 

isdividuai embedded code signals. This entails the storage of a significam amount of additional code 

Lnfonnanon above and beyond die anginal, and many appiicadons may meni some foim of economizing. 

One such approach to economizing is to have a given set of mdividuai embedded code signals be 

common to a batch of source maceriats. For example, one diousand images can all milize the same basic 

20 set of individual embedded code signals. The storage requirements of diese codes dien become a small 
ftacdon of the overall storage tequirements of the scnizce m a t e r ial. 

Fonfaermore, some applications can iiHIiti^ a, univenal set of individual embedded code signals, 
i.e., codes whicb remain die same for all instances of distribmed material. This type of requirement would 
be seen by systems which wish to hide die N-bit identi&cation word itself, yet have standardized equipment 

25 be able to read dxai word. This can be used m systems which maJce go/no go decisions at potnt-of-reail 
locations. The potential drawback to this set-up is that die universal codes are more prone to be sleuthed 
or stolen: dtercforc dicy will not be as secure as die apparams and methodology of die previously disclosed 
arrangement. Perhaps diis Is jiBt die diffierence between 'high secnricy' and 'air-dght security/ a 
distinction caiiying' little weight with the bitt of potential applicatiotis. 

30 

TUi. m Pmiring. Pmer. Dociimeiits. Plasti c Coated Tdentificarion Cards, and Other Material Wbgye G^(^a^ 
Embedded Codes Can Be Imprinted 

The term 'signal' u often used narrowly to refier to digital data mformation, audio signals, 
images, etc. A broader imeiprctaiion of 'signal/ and die one more generally intended, inchidcs any form 
35 of modttiacion of any mamrial whatsoever. Thus, die micro-iopolpgy of a piece of common papci becomes 
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a 'signal' (e.g. it hcisht as a fenciion of x-y coordinaccs). The reflective properties of a flat piece of 
piasdc (as a ftanctioii of space also) becomes a signal. Tlie poini is chat photographic em ui si o n s. audio 
signals, and digtixzed mfonoatiott ace sot the only types of sigoaU capable of utiliziag the prmciples 
described herein. 

5 As a in point, a irar^inr^ very mudi resembling a biaiiie primiiig niai'Mitr can be desi g ned so 

as CO imprint unique 'aoise4ike' indentations as outlined above. These m rfeniari om can be applied with a 
pcessQit wiiich is mncb smaller than ia typically applied in creating braille, to dte point where the patterns 
are not nodced by a nonnai user of the paper. Bui by following the steps of the present disclosure and 
applying them via the mecbanism of micro4ndentatioiis, a tmique identification code can be placed onto 

10 any given sheet of paper, be ii intended for everyday stationary purposes, or be it for important 
documcnis, legal tender, or other secured material. 

Tlie leading of the klentificadon material in such an embadiment generally proceeds by merely 
reading the docoment optically at a variety of angles. This would become an inexpensive mediod for 
deducing the micro-topology of die paper surfecc. Certainly odicr forms of reading die topology of che 

15 paper axe possible as well. 

la the case of ptasuc encased material such as identification cards, e.g. driver's licenses, a similar 
braiHe-Iike impressions machine can be utilized to imprint unique identification codes. Subtle layers of 
pbototeaoive materials can also be embedded iwidc the piasdc and 'exposed.' 

It is clear that wherever a material exists which is capable of being moduiated by 'noise-like' 

20 signals, that macoial is an appropriate carrier for unique idendfication codes and utilizadon of the 
princi^ies disclosed herein. All diat remains is che matter of economically applying die idemtficadoa 
fflfonnation and ^• r*^^*^^ die signal level below an acceptability cfazeshold wbidi eodi and every 
aj^lication will dcfioe for itself. 

25 REALTIME ENCODER 

While die fim class of embodiments most conunonly en^Ioys a standard microprocessor or 
compraer to perfram die encodation of an image or signal, it b possible to utilire a custom encodatkm 
device which may be ftster dian a typical Von Ncuman-cype processor. Sudi a sysmm can be udlizBd wdi 
all msoner of scxial'dain streams. 

30 Music and videotape recordings are examples of serial datt streams - data streams which are 

often pirated! It would assist enforcement efforts if audmrized recotdmgs were encoded wirii idcntificarion 
data so dut pirated knodc-ofb could be ctaceti to the origiiial turn whidi they were made. 

Piracy is but one concern driving die need for i^Ucant's tedmoiogy. Another is ariTh r mi ca ifo n, 
Often it is important ro amfirm that a given set of data is really what it is purported to be (often several 

35 years after its generation). 
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To address these and other needs, the system 200 of Fig. 5 can be employed. SysKm 200 can be 
choughc of as an 'yi ^f^firsainn coding black box 202. The system 200 receives an inpot signal (sometinies 
termed the 'tnaster" or "unencoded' signal) anil a code word, and produces (generally in real time) an 
idemiiicacion-coded output signal. (Usually, the system provides key data for use in laier decoding.) 
5 The contents of the "blade box" 202 can take varioos forms. An exemplary black box system is 

shown in Fig. 6 and induden a look-up table 204, a digital noise aooice 206. first and second scalers 208« 
210, an adder/subtracier 212. a memory 214, and a register 216. 

The input signal (which in the illu^rated emfaodimeoc is an 8 - 20 bit data signal provided at a rate 
of one million samples per second* but which in other embodiments couid be an analog signal if 
10 appropriate A/D and D/A conversion is provided) is applied from an input 218 to die address input 220 of 
the k)ok-up table 204. For each mput sample (Le. look-up table address), the table provides a 
coirespondmg 8-bit d^itaJ ouQ»t word. This output word is used as a scaling ^tor diat is applied to one 
input of die fusr scaler 208. 

The first scaler 208 has a second u^nit, to which is applied an 8-bit digital ooise signal &om 
15 source 206. (In the illustrated embodiment, the noise source 206 comprises an analog noise souice 222 
and an aoaiog-to-digital convener 224 although, again, odicr implementations can be used,) Ihc noise 
source in the illustrated embodiment has a zero mean output value, witii a full width half maximnm 
(FWHM) of 50 - 100 digital numbers (e.g. from -75 to +75). 

The fint scaler 208 multiplies the two 8-bit words at its inputs (s«de factor and noise) to produce 
20 - for each sampte of the system input signal - a 16-bit output word. Since the aoisc signal has azcto 
mean value, the output of die fost scaler likewise has a zero mean value. 

The ouipmofthe fint scaler 208 is applied to the input ofthe second scaler 210. The second 
scaler serves a gtobal scalhig function, establishing the absolute magniwde of the identification signal that 
will ultimately be embedded into die hiput data signal. The scaling facwr is sec dirough a scale control 
25 diBvice 226 (which may take a nun^ of forms, from a simple rheostat to a grapMcaily implemented 
control in a graphical user inicrfecc), permitting this factor to be changed in accordance with the 
requiiements of diffecn: applicaiions. The second scaler 210 provides on its output line 228 a scaled 
noise signal. Each sample of this scaled noise signal is successively stored m the memory 214. 

(In dxe ilhistratcd embodiment, djc output ftom the first scaler 208 macy range between -1500 and 
30 +1500 (decimal), white the ouqjut from dm $ea»d scaler 210 is in dm tow single digits, (such as bttwecn 
-2 and +2).) 

Register 216 stores a muld-biiidentificatKm code word. In the illustrated embodiment diis code 
word consists of 8 bits, aldmu^ larger code words (up to hundreds of bits) arc commonly used. These 
bits are referenced, one at a dn», to control how the faiput signal is modulmed with the scaled noise signal. 
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In panicular, a pointer 230 is cycled sequennally thiDUgh cbe bit posidoos of the code word in 
register 216 to provide a control bit of "O* or T lo a contzol input 232 of the addsr/sabnacier 212. If. 
for a particular input signal sample, the control bit is a tbe scaled noise signal sample on line 232 is 
added to the input signal sample. If die omtzol bit is a "0*, cbe scaled noise signal sample is snbtcacted 
5 from ±e input signal sample. The ouiput 234 from ibe adder/subtracter 212 provides dxc blade box's 
output signal. 

The addidoo or sitoacxion of ibe scaled noise signal m accordance with die bits of the code woid 
effects a moduladcsi of die b^ signal that is generally ic^erceptible. However, widi knowledge of cbe 
contecis of die memory 2I4» a user can later decode tbe encoding, detcrminisg die code number used in 
10 die original r""^i"g process. (Actually^ use of memory 214 is optional, as explained bdow.) 

It will be recognized thai the encoded signal can be distributed in well known ways, including 
oonvened to printed image forsu scored on magnecie media (floppy diskette, analog or DAT tape, etc.), 
CD-ROM, etc. etc. 



15 Decoding 

A variety of techniques can be used to determine the identificaiian code with whidi a suspca 
signal has been encoded. Two arc discussed below, Tlie fiisi is less preferable than the latter for most 
plications, but is discussed herein so that the reader may have a fuller conte^ within which to 
understand die disclosed tec hn o logy, 

20 More particularly, die first '^'^^^f^ method is a difference niEtkxi, relying on subtraction of 

Gorresponding samples of the original signal from die sospea signal to obtain difference satnples. which 
are dien examined (typically individuaUy) for deterministic codmg mdicia (i.e. die stored noise data). This 
approach may thus be texxned a 'sample-based, detennimstic" decoding technique. 

The second ^♦'■^^"g mediod does not make use of the original signal. Nor does it examine 

25 particular samples looking for predetermined noise dxmcienstics. Rather, the siatistics of the suspea 
signal (or a portion diereof) are considered in die aggregate and analyzed to discern the presence of 
identification coding diat permeates the entire signal. Tbe reference co permeation means the entire - 
idonificacioa code can be discerned from a small fragment of die suspea signal. Hus latter approach msy 
thus be Tf™^ a "hblograpbic, statisdcai" deoodmg tectanique. 

30 Bodi of diese mediods begin by registering tbe suspect signal to match die origmal. This entails 

scaling (e.g. in amplimde, duration, color balance, etc.). and sampling (or resampling) to restore tbe 
origmal sample rate. As m die eatiier described embodimmT, there are a varieiy of weil understood 
techniques by which die operations associated with this registration function can be performed. 

As noted, the fint decoding approach proceeds by subtracting the original signal from the 

35 r^stered, suspea signal* leaving a di^erence signal. The poiariiy of successive difference signal samples 
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cm duQ be compared with cbe paiarides of tbe corxcspondiog stored noise stgsai samples to detenmoe die 
identificaiias code. ThsL is, if the polarity of die first diffiercnce signal sample matches diat of ±6 fint 
aoise signal sample, then Ehc fiisc bit of die icfeadfication code is a (la sudi case, die polarity of die 
9di, 17di, 25±^ etc. samples should also all be positive*) If die polarity of the first diffiexencs signal 
5 sample is opposite diat of die corresponding noise signal sample* dicn die first bit of die idemificatian code 

By conducting die foregoing analysis with e^hi successive samples of die difference signal, die 

sequence of bits diat comprise the original code word can be detexmined. If, as in die iUusuated 

embodiment, pointer 230 stepped through the code word one bit at a time, beginning with die first bit, 
10 during encoding, dien die first 8 samples of die difference signal can be analyzed to miquely determine die 

value of die 8-bic code word. 

In a notse-fiee world (^leaking here of noise independent of diat with which the identification 

coding is effected), die foregoing analysis would always yield die oorrect ideotincatiQn code. But a 

process diat is only applicable in a noise-fiee world is of limited utOity indeed. 
15 (Further, accurate identification of signals in noisc-frce contexts can be handled in a variety of 

odier, simpler ways: e.g. checksums; statistically improbable correspondence between suspect and original 

signals; etc.} 

While noisc-indoccd aberrations in decoding can be dealt with - to some degree - by analyzing 
large pordons of die signals such aberradons still place a practical ceilhig on die confidence of die process. 

20 Funher* die villain that must be confronted is not always as benign as random noise. Rather, it 

increasingiy takes the fonn of human-caused corruption, distortion, manqndation, etc. In sudi cases, the 
desired degree of idemificaiioa confidence can only be achieved by other approaches. 

Hie jllusitated embodiment (die "holographic, statistical** decoding technique) relies on 
recombining die suspect signal with certain noise data (typically die data stored in memory 214), and 

25 analyzing die cnnopy of die reciting signal. "Entropy* need not be understood in its most stria 

mathemancai definidon, it being merely the most concise word to describe randomness (noise, smoodmess, 
snowiness, etc.)* 

Most serial dau signals are not random. That is* one sample usually correlates - to some degree 
^ widi the adjacem'sampies. tioise^ in contrast* ^xcally is random. If a random agnaf (e.g. noise) is 
30 added (0 (or subtracted fitoffl) a son-random signal, die entropy of die resoidng signal generally mcieases. 
That is, die resulting signal has more random variations dian die original signal. This is die case widi the 
encoded output signal produced by die present encoding process; it has more entropy dian die mginal, 
imenexidfd signal. 
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If, in coQtrasc, the addidon of a laodom sigoai vo (or sobtracdon fi[om] a ooa-candom sigml 
rednces entropy, then somedung loiusaal is happening, {t is this anomaly that the ptescnt decoding process 
uses to detect embedded ktencificatioa coding. 

To My understand diis entropy*based decoding method, it Is first helpful to higfaii^t a 
S characteristic of die original Gicoding process: the similar cieatrnmr of eveiy eighdi sample. 

la the encoding process discussed above» the pointer 230 incicmaxts through die cade wonU one 
bit for each successive sample of die input signal. If the code word is eight bits m length, (hen die pointer 
retUEis to the same bit posidan in die code word eveiy eighth signal sample, [f dus bit is a 'T, noise is 
added to the hipuc signal; if this bit is a "0*, noise is subtracted froin the input signal. Due to die cyclic 
10 progression of Che pointer 230, every eighth sample of an encoded signal dius ^laies a characteristic: they 
are all eidier augmented by the corresponding noise data (which may be negative}, or diey are all 
dnniniahed, d^ending on whether die bit of die code wotd then being addzessed by pointer 230 is a T or 
a"^". 

To Gtploh this characterisdc, the entropy-based deco<^g process treats every eighth san^le of the 
15 suspea signal in like fiishion. In particular, the process b^ins by adding to die lst» 9th. 17di, 25di. etc. 
saxniries of the suspea signal die corresponding scaled noise signal vahiea stored m the memoty 2U (Le. 
diosc stored in the 1st. 9di, 17di. 25di, etc., memory iocadons, respectively). The entropy of the resulting 
signal (Le. die suspect signal with every 8di sample modified) is then cmnputed* 

(Conqiutadon of a signal's ennopy or randomness is well understood by artisans in diis field. 
20 One generally aoospvsd tedmiqoe is to take die derivadve of the signal at each sample point, square these 
values, and dien sum over the entire signal. However, a variety of other well known techniques can 
alternadvety be used.) 

The foregoing step is dien rqieated, diis dme subtracting die stored noise values from the 1st, 9d2, 
17du 25 etc. suqm signal samples. 

25 One of diese two operadons will undo die encoding process and reduce die resultmg signal's 

entropy; the odier win aggravate it. If adding die noise data in memory 214 to die su^ea signal reduces 
its entronv. diea diis data mast eaiiier have been subtracad from dig oiiffinai signal. This indicates diat 
pointer 230 was poindng to a '0* bit when diese samples were encoded. (A "0* at the control input of 
addex/auhtraeter 212 caused it to subtract the scaled xioisc fc90k die iiqnit signal.) 

30 Convexsely, if subtracting the noise data from every eighdi sanqde of the suspect signal reduces its 

entropy, dien the encoding process must have earlier added this noise. This nidicaifts diat pointer 230 was 
ptnndsg (0 a *r bit when sasqiles 1, 9, 17, 2^, etc., were encoded. 

By noting whediff entropy decreases by (a) adding or (b) subtracting die stored noise data to/from 
die suspect signal, it can be determined diat die first bit of die code wtxd is (a) a 'O', or (b) a *I *. 
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The foregoing opennoos ayre thai condncced toe the group of spaced samples of the suspect signal 
begiimiag with the second sample (Le. 2, 10, 18, 26 ...)* 'Hie entropy of lbs lesuliing signals indicate 
whether the second Int of the code vrard is a *0" or a *r. Likewise with ±e Mowing 6 groups of 
spaced san^ks in the suspea signal, nntil all 8 bits of die code word have been discerned. 
5 It will be appreciated that the foregoing ^proacfa is Dot sexisitive to corxvption mecfaanisou that 

alter die vahies of individual samples: instead, the process cansidcrs die entropy of die signal as a whole, 
yielding a high degree of confidffnrc in the resides. Ftmfaer^ even small excerpts of the signal can be 
analyzed in this manner, penmning piracy of even small details of an original worJc to be detected. The 
results are dms statisticaily robust, both in the face of natural and human comiptiaQ of die suspect signal. 

IQ li win ftinher be appreciated diat the use of an M-bit code word in this real time embodiment 

provides benefits analogous to those discussed above in connection widi die batch encoding system. 
(Indeed, die i»esent emboditnent may be ooncepiualized as making use of M diftoni noise signals, just as 
in the batch encoding system. The Srsc noise signal is a signal having die same extern as die input signal , 
and compnsxng die scaled noise signal at die 1st, 9th» 17th« 25th. etc, samples (assuming N =8), widi 

15 zeroes at die intervening samples. The second noise signal is a similar one comprising ihc scaled noise 
ygwar at die 2d, 10th, 18th, 26di, etc.. samples, with zeroes at the intervening samples. Etc. These 
signals are all combined to provide a composite noise signal.) One of the Lmportaat advantages inherent m 
such a system is die high degree of siaasdcal confidence (confidence which doubles widi each successive 
bit of die identificaiion code) that a match is really a match. The system does nol tdy on sid>jective 

20 evaluation of a suspect signal for a single, detennmistic embed d ed code signal. 



10 



miimtive Variarions 

From die fciegoing dexripiion, it will be recognized diat numerous motfifications can be made to 
die illustrated systems wifliout changing die f^mdamental principles. A few of diese variations are 
described below. 

The above-described decoding process tries bodi adding smd subffacting stored noise data to/Efom 
the snspea signal m order to find wtuch operation reduces enocpy. In other embodiments, only one of 
dicae operations needs to be conducted. For example, in one alwmaiive deoxiing process die stored noise 
data corresponding to every eighdi sample of dw suspect signal is only adde^ » said samples. If the 
entropy of die resulting sigma is dieaby inrxcased, dien die coiTespanding bit of dm code word is a T 
a. e, dns noise was added earlier, during die encoding process, so adding it again only compounds die 
signal's randomness). If die entropy of the tesultiag signal is dietcby dowased, dien die correspo^ 
bit of die code woni is a '0". A fhnher rest of encopy if die stored noise samples are fflb^ff^at is not 
leqnited. 
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me suosticai reliability of (he identification process (coding and decoding) can be designed to 
exceed viitually any confidence diresiiold (e.g. 99.9%, 99.99%, 99.999%. etc. confidence) by appropiiatc 
selecdoa of die global scaling facmn, etc. Additional confidence in any givea application (unnecessaiy in 
won ippitotioos) can be achieved by recfaecking the drroritag process. 

5 One way to techedt die decoding process is to remove the stored noise dm fiom the suspect 

sigoai in accordance widi die bits of Ae discetned code word, yielding a "testored" signal (e.g. if die first 
bit of die code word is found to be *!,* then die noise samples stored in the Ut, 9th, I7di« etc. locations 
of the memory 214 are sobtracied from the conesponding samples of the suspect signal). The entropy of 
the tcstored signal is measured and used as a baseline in funher measurements. Next, the process is 

10 repeated, this time removing die stcrcd noise data from the suspect signal in accordance wiih a modified 
code word. The modiffcd code word is die same as die discerned code word, except 1 bit is toggled (e.g. 
die first). The entropy of die resulting signal is determined, and compared widi die baseline. If die 
toggling of die bit in die discerned code word resoited in increased entropy , dien the accuracy of diat bit of 
the discerned code word is confirmed. The process repeals, each dme with a difPereni bit of die discetned 

15 code word toggled, until all bits of die code word have been so checiced. Each change should result in an 
increase in entropy conipared to the baseline value. 

The data sioicd in memory 214 is subject to a varicry of aliernaiives. In die foregoing discussion, 
memory 214 contains die scaled noise data, la other embodiments, die unsealed noise data can be stored 
instead. 

20 In still odicr embodiments, it can be desirable to store at least pan of die input signal itself in 

memory 214, For example, die memory can allocate 8 signed bits to die noise sample, and 16 bits to store 
die most significani bits of an 18- or 20-bic audio sigoai sample. This has several benefits. One is dm it 
simplifies rcgistiaiiott of a "suspect" signal. Anodier is dial, in die case of encoding an input signal which 
was already tnr"**"^, the data m memory 214 can be used to discern which of die en ood in g processes was 

25 pcrfbrmed first. Thai is, ftom dw mpm signal dam in memory 214 (albeU mcomplcie), it is generaUy 
possible to detennme widi which of two code words it has been encoded. 

Yet anodicr alternative for memory 2U is diat is can be omitted altogether. 
One way diis can be achieved is to use a determinisdc noise source in die encoding process, sudi 
as an algorithmic noise generator seeded widi a known key number. The same detemunisdc noise source, 

30 seeded widi die same key anmbcr, can be used in die decoding process. In snch an anrangcmcm, only die 
key number needs be scored for later use in decoding, instead of die large data set usuaUy stored m 
memory 214. 

Alternatively, if die noise signal added daring eocodmg docs not have a zero mean value, and die 
leogdi N of dw cocte word Is known to die decoder, then a universal decoding process can be impiemented. 
35 This process uses die same entropy test as die foregoing procedures, but cydcs dirough possible code 
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woRis» addii^sutoaciifig a smaU dummy notse value (e.g. [ess dus the expected mean noise value) lo 
every Nth sample of die su^ect signal, in accordance with die bits of die code word being tested, until a 
reduction in ennopy is noted. Such an approach is not favored for most applications, however, because it 
offers less security than die o±er eod^odiments (e.g. it is Aibjea to craddng by brme force). 
5 Many <^licadons are well served by die embodiment illustrated in Fig. 7, in which different code 

words are used to produce several differendy encoded versions of an input signal, each making use of die 
same noise data. More particuiariy. die embodimenr 240 of Fig. 7 inchides a noise store 242 into which 
noise fiom source 206 is wciitcn during the kleniificaiion*cadtaig of die input signal whh a fixst code word. 
(The noise source of Fig. 7 is shown outside of the real time encoder 202 for convenience of iilustradon.) 

10 Thereafter, additional identification-coded venions of die input signal can be produced by reading die 
stored noise data Cram the store and using it in coiijunaion with second through Nth code words to encode 
the signal. CWlixle binajy-sequentiai code words are illustrated m Fig. 7, in other embodiments arbitrary 
sequences of code words can be employed.) With such an arrangemem, a gxear number of diffezendy- 
encoded sigonls can be produced, widiout requiring a pioportiQnally>sized long term noise memory. 

15 Instead, a fixed amount of noise data, is stored, whether encoding an original once or a drnmand dmes. 

(If dedred, several differendy-coded output signals can be produced at die same time, radier dian 
seriatim. One such implementation inchides a plurality of adder/sidstracier circuits 212, each driveo with 
die same mpuc signal and widi die same scaled noise signal* but with different code words. Each, then, 
prodnces a difbently encoded output signal.) 

20 In applications having a great number of differendy-cncodcd versions of the satne original, it will 

be recognized dtat die decoding process need not always discern every bit of die code word. Sometimes, 
for example, the plication may require identifying only a group of codes to which the suspect signal 
belongs. (E.g., high order bits of the code word might indicate an organizarioa to which several 
differently coded venions of the same source material were provided, with low-orcfer bits identifying 

25 . specific copies^ To identify die organizatian with which a suspect signal is associated, it may not be 
necessary to exanoine the low order bits, since the organizadon can be identified by die high order bits 
aionc.) If the identi&adon requirements can be met by discerning a subset of the code word bits in the 
suspea signal, the tfywitwg process can be shortened. 

Some ai^cacians may be best served by restarting die encoding process - sometimes with a 

30 different code wonl - several dxnes widiin an integral work. Consider, as an example, videotaped 

ptoducdofls (e.g. television programming). Each frame of a videotaped production can be Ideniificatioi^ 
coded widi a unique code mimber, processed ui re:d*time with an arrangement 24S lilce dial shown hi Fig. 
8. Each dmc a vertical retrace b detected by sync dctcaor 250, the noise source 206 resets (e.g. to repeat 
die setpience just produced) and an identificatiQn code mcxmnems to the next value. Each irame of the 



SUBSTmiTE SHEET (RULE 26) 



wo 97/43736 



PCT/US97/0B351 . 



-28- 

videot^ is dieieby uniquely ideatillcaiioii-coded. Typically, tbe encoded signal is stored oa a videotsfw 
for long cenn storage (alttough other storage media, indoding laser disks, can be usoQ, 

Etenmnng to die encoding apparatus* die look-up table 204 in die illiistrated emfaottiment e:q>loics 
dtt ftasc dxat higlt aniplimde samples of the hi^ut data signal can tolerate (without objecttooable degradation 

5 of die ouspui sigoaO a higher level of mnoderi identification coding diaa can low amplinide input samples. 
Thus, for example, inpm data samples having decimal values of 0, 1 or 2 may be oorrespond fm the look* 
up table 204) to scale factors of unity (or even zero), whereas input d^ samples having values in excess 
of 2CX) may coitespond to scale factors of 15. Generally speaking, die scale factors and the iapoc sample 
vafaiea amcspond by a square root relation. Thar is, a four-fold increase in a value of the sampled input 

10 signal corresponds to apptoximateiy a cwo-foid increase hi a value of the scalhq factor associated 
therewith. 

(Tbe parenthetical r eferen ce to zero as & scaling fiictor alludes to cases. e,g.. m which the source 
signal is temporally or spatially devoid of informatioo OHitent. In an image, for example, a region 
characterized by several contiguous sample values of aero m^ eonespood to a jet black region of ±e 
15 frame. A scaling value of zero may be appropriaie here since diere is essentially no image data to be 
pirated.) 

Comhnixng with the encoding process, those sicilled in the art will recognized ilie poieniiai for 
"rail errors'' in die illustraied emhodimenc. For example, if the uiput signal consists of 8-blt samples, and 
die sampiea span ths entire range &om 0 to 255 (decixnal)f duo the addition or sobtzacdon of scaled noise 

20 to/ftom the input «gnal tssf prodnce output signals diat cannot be represented by 8 bits (e.g. '2, or 2S7). 
A ^wwhPf of well-undersiood t^^t " ^ exist to rectify dus situation, some of them proacnve and some of 
&ft^ reactive. (Among diese known technicpies are: specifying that die input signal shall not have samples 
m die range of 0-4 or 251*255, thereby safely permitdng modulatiott by die noise signal; or including 
provision for ^^'^^z ^ atiaptively modifying input signal samples that wodd otherwise cause tail 

25 erxQR,) 

While the iliustraced embodiment describes stepping duough die code word sequentialiy, one bit at 
a dmc. to control modulation of successive bits of die input signal, it will be appreciated diat die bits of 
die code word can be used odier than sequentiaily for this purpose. Indeed, bits of the code word can be 
selecsed m aocoxdance with any ptedemnmed algoiidun. 
30 ' The dynamic scaling of die noise signal based on die instantaneous value of the input signal is an 

opdmizadtm diat can be ondtted in many embodiments. That is, die lookmp table 204 and die first scaler 
20S can be omitted emiiely, and the signal from die digital noise source 206 applied direcdy (or dirough 
die second, ghibal scaler 210) to tbe add^subtracter 212. 
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b win be finther iccognized that the use of a zero-meaa noise source siinpiifies the Uhistrated 
anbodimect, but is not essemUI. A ooise signal with another mean value can zeacfily be used, and D.C. 
compen^OQ (if aeeded) can be effected elsewhere in the system. 

The use of a noise sounx 206 is also opiionai. A variety of other signal sauioea can be used, 
5 depending on appUcanon- dependent constraints (e.g. the threshold ac which the cnoided idendficadcn 
signal becomes pexccptxbic). In nsny ustanccs, the level of (he embedded identification signal is low 
enough ihac die kientiScation signal needn't have a random aspect; it is imperceptible regardless of its 
natore. A psendo random source 205, however, is usnatty desired because it provides die gxeatest 
idcnrificatinn code signal S/N rado (a somewhat awkward teem in this instance) for a level of 
10 tmp e iLep tibi]ity of die emheriftari idetuification signal. 

It will be recognized that identification coding need not occnr a&r a signal has been reduced to 
stored fioim as data (i.e. 'fixed in tangible focm,* in die words of the U.S. Copyright Act). Gsnsider, for 
exan^le, die case of popular mnsirians whose performances are often recorded illicitly. By identification 
coding the audio before it drives concert hall speakers, unandiorized rccordtogs of die concert can be 
15 traced to a particular place and cim& Ukcwise, Uve audio sources such as 911 emergency calls can be 
encoded prior to reo^ding so as to fariliratpt dxeir later andsendcation. 

While the black box embodiment has been described as a stand ahxie unit, it will be recognized 
that It can be huegraied into a number of di£Bsent tools^nstnxmenis as a con^onent. One is a scanner, 
which can embed identification codes m the scanned output data. CHie codes can simply serve to 
20 memorialize diat the data was generated by a pardcular scanner). Anodtei is in creadvliy software^ soch 
as popular dxawing/graphics/animatian/paint programs offered by Adobe, Macromedia, Corel* and the like. 

FinaUy, whUe die real-time encoder 202 has been ilhistraced with reference to a paiticoiar 
hardware hnplementaiian, it will be recognized ±ai a variety of other implementadons can altematively be 
ensplt^ed. Some utilize odier hzdware configuradons. Others make use of software routines for some or 
25 ail of the illustrated funcdonal blocks. (The software routines can be ctecutcd on aoy number of differcm 
general purpose programznablc computeis, such as 30x86 PC-cnmpatifaie cocqnireis, AISC*based 
warkstations, etc«) - - 



TYPES OF NQBR OUASI-NO ISE. AND OPTTMIZEP-NOISE 
30 HeiciofDre dus disclosure posnilated Gausiaa noiset 'white noise/ and noise general 

bom application tnstzumeatation as a few of the many examples of the kind of carrier signal appre^riate to 
carry a single bit of mformation throughout an image or signal. It is possible to be even more proactive in 
^designing* dsaraoeiistics of Ksise in ortier to achieve certain goals. The 'design' of using Gaussian or 
instrumental noise was aimed somewhat toward "absolute* security. This section of the disclosure takes a 
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10 



look ai ocber caosideratioos for ibe design of ihe noise sigcais which may be Gonsidered die ulciaiare 
cairicn of die idcatificalioii infonsatioa. 

For some appiicadoas might be advantageous to design die noise carrier signal (e.g. the Ndx 
embedded code signai in die first embodimeas; the scaled noise data in die 3econd embodinient). so as to 
provide moie absolute signal strengtfi lo die identificatioQ signal lelative to die peicepcibiliiy of tba signal. 
One example is the Mowing. It is recc^nized dxat a tne Gaussian noise signal has the value '0' occur 
most fteqaoaly. followed by I and -I at equal probabilities to each odicr but lower than '0', 2 and -2 
next, and so on. Qearty, the value zero carries no infonnadon ai it is used in aicfa an embodiment. 
Thus, one simple adjustnieor, or design. wouW be thai any time a zero occurs in the generanon of die 
.»m>wiri>^ code signal* a acw process talccs over, whereby die value is convetted "randomly" to eidier a 1 
ora-L In logical terms, a decision would be made: tf'0\ dam random(l,-l)- The histogram of such a 
piocess would ^ear as a Ganssian/Poissonian type disaibudon, excqit that die 0 bin would be empty sad, 
the 1 and -1 bin would be increased by half die usoal histogram vahie of dtf 0 bin. 

In diis case, fdrnrifi* ?^ signal energy would always be applied at all parts of die signaL A few 
15 o£ die trade-ofifc include; dierc is a (probably ncgligiWe) lowering of security of die codes in diai a 
•deteBnimstic compooenf* is a pan of gencranng dse noise signal. The reason diis might be completely 
ficgligibic is dtfl we still wind up widi a coin flip type situation on randomly dioosing the 1 or die -l. 
Anfldicr iradeoff is dial diis type of designed noise will have a higher threshold of perceptibility, and wiU 
only be applicable to applications where die least significant bit of a data rtrttans or image is already 
negligible xdadve to die commercial value of die material i-e. if die least signiScant bit were stripped 
bom die fignal (for all signai samjto). no one would know the difference and die vahic of die material 
would not suffer. This bloddng of d» zero value hi the example above is but one of many ways to 
•optimize* die noise properties of die signal carrier, as anyone m die an can realize. We refer to diis also 
as 'qoasi-noise' m die sense diat namral noise can be transfonned in a pte-detennined way into signals 
25 whidi for aU intems and purposes wUl read as noise. Also, oyptogr^shic mcdiods and aigoriduDs can 
easily, and otten by defittition, crtate signals which are perceived as completely random. Thus die word 
•noise' can have different connotations, prinaarily between diat as defined subjectively by an observer or 
listener, ami diat defined madicmaricaUy. The difference of die Utter is diai maihwnatical noise has 
difieratpropcmes'of security and die simpUdcy^ ii can ddicr be "sleudied' or die simpUcity 
with wfaidi umtumcms can 'automatically xecogniae' the existence of this noise. 



20 



30 



nJniv»Mi' Embedded Codes 

The bulk of dus disclosure teaches Uiai for absolute security^ die noise-lite embedded code signals which 
cany die hits of information of ^ ideciification signai should be unique to cadi and every encoded signal, 
35 or, slightly less restrictive, diat embedded code signals should be genenied sparingly, such as usmg die 
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same *T nMH«< codes for a batch of 1000 pieces of fUnu for ezaaple. Be this u it may. ihcre is a whde 
ocber ^pfoadi to tbis Issue wbeicizi the use of what we wiU call 'uiiiveRai* embedded code signali can 
open up large aew appiicatiDu for this technology. The economica of diese uses would be such diat die de 
facto lowered security of these universal codes {e.g. they would be analytabie by time honored 
5 cryptographic decoding methods, and thus potentially thwarted or reversed) would be economicaiiy 

negligible relative to die econcnnic gaios Uiat the intended uses would provide. Piracy and illegidmate uses 
would become merely a predictable "cost" and a source of uncollected revenue only: a simple line item m 
an ccononnc analysis of the whole. A good analogy of this is in die cable mdusiry and the scxamblmg of 
video signals. Everybody seeau to to»w that crafty. sldUed technical uidlvidaais, who may be generally 

10 law abidmg citizens, can climb a ladder and flip a Sew wires in dicir cable junction box m order to gel all 
the pay d iq mtf!* for liee. The cable mdustry knows diis and uices acdve measures oo stop it and prosecute 
diose caught, but the "lost revenue' derived from this practice remains prevalem but alnmst nc^ 
pcrccmagc of profia gained from die scrambling system as a whole. The scrambling system as a whole is 
an economic sucf^^ despite its lack of "absolute security.* 

15 Hie same holds true for applications of this technology wherein, ftw die price of lowering security 

by some amount, large economic opportunity piesems itself. Hiis section first describes ^stat is meant by 
undvetsal codes, then xnovcs on to some of the interesting uses to which dxcsc codes can be applied* 

Universal embedded codes generally refer to the idea Uiat knowledge of the exact codes can be 
distributed. The embedded codes wwi't be pm into a dark safe never lo be touched untU litigation 

20 (as alhidcd to in odier parts of diis disclosure), but instead wiU be disttibuted to various locations where 
ott-die'5pot analysis can cake place. GencraUy this distribtrtion win still take place withm a security 
controlled environment, meanmg diat steps will be taken to limit the knowledge of die codes to diose with 
aneedtokaow. instrimicniation whidi atteinpts to automaticaDy detea copy 
example of "somcdiing* with a need to know die codes. 

25 Tliere are many ways to implement the idea of universal codes, each with dien- own merits 

regarding any given application. For the purposes of teaching this an, we xpaait diese approaches into 
dnee broad categories: universai codes based on UTMaries, universal codes base^ 
and imiversal codes based on iw-dcfined industry standard pai^ A rough role of dwmb is dia the 
tot is mote secure dm die laner two, but tbat the latter two are p^^ 

30 dmn die Sist. 

iTnivgrsai Codes: 0 T jTifaries of Universa] Codes 

The use of libraries of universal codes simply means that applicant's techmqucs are employed as 
described, except for die fact that only a limited set of die incfividual embedded code signals are generated 
35 and that aro? given encoded material will make use of some sub-s« of this liariied •universal set/ An 
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exan^jlB is in order hoe. A photograpUc prim paper manufacnircr may wish m prc-cxposc every piece 
of 8 by 10 tncii print pap^ which chey sell with a unique identiScatkm code. They also wish to sell 
idmtificaciott code leoogniiioa software to their laxg9 custcmiers, service biutaus, stock agcndea, nd 
individual photographers, so that all diese people can not only verify that their own material b corrrcdy 

5 oiaitaL but so that tbey can also detennme if diird parry material which they are about u> acquire has 
beea r^»ff*d by diis tecfaitology as being copyrighted. This latter mfortnation will help them verify 
copyright hoiden and avoid lidgatioa. ant»ig many odier benefits, hi order to "eanunmcaily" insdnite 
this pian^ chcy tealizc dot gcneranng uziique individTial embedded codes lor each and every piece of print 
paper would generate Terabytes of uidepeodent infiannation, which would need scoring and to which 

10 recognition software would itecd access. Instead, they decide to embed their print paper with IS bit 
ideniificaiion codes derived from a set of only 50 mdependent *uaiversai* embedded code signals. The 
details of how dus is done are m the next paragraph, but the point is that now their recognition sof^aie 
only needs to contain a linnjed set of embedded codes in dieir library of codes, typical^ on die order of 1 
Megabyte to 10 Megabytes of mformatioo for 50x16 individual en^edded codes splayed out onto an 8x10 

15 photographic print (allowing for digital compression). The reason te picfcnig 50 mstcad of jusi 16 is one 
of a little mot* added security, where if it were die same 16 embedded codes tor all phoiogr^hic sheets, 
not only would die serial mnnber capabiiicy be limited to 2 lo the i6ch power, but lesser and lesser 
sophisdcaicd pirates could crack the codes and remove them using softwae toob. 

Thcic are many differem ways to implement this scheme, where dje following is but one 

20 exemplary method, it is deieimhied by die wisdom of oompaixy managcmem dm a 300 pixcU per ioh 
criteria for die embedded code signals is snifident resolntzoo for most applications. This means that a 
composite embedded code image wiU contain 3000 pixels by 2400 pixels to be exposed at a very low levd 
onto each 8x10 sheet. This gives 7.2 million pixds. Using one siaggetcd codmg system such as described 
m the W -'*"*' box hnplementadon of Figs. 5 and 6, each individual e m fa e d drri aide signal will contain only 

25 7.2 nsHion divided hy 16, or approximately 45QK uue inferoaiion carrying pixels, every I6di pixel 
along a given rastw line. These vahies will typically be in die range of 2 to -2 in digital numben, or 
adequately dcscrihed by a signed 3 bit onmber. The raw mfbnnation content of an embedded code is then 
approximatdy 3/«m*s bytes dmes 450K or about 170 Kilobytes, Digital compressian can reduce tfiis 
funfaer. AD of diwdecisiona arc subject to standard engmceiingopiimiaaM 

30 any given sqipiicaiion at hand, as is well known hi die art. Thus we fed diat 50 of these mdependent 
embed<fed codes wiD amount to a few Megabytes. This is quite reasonable level to distribute as a •library" 
of uttiveisal codes widim the recognidoa software. Advanced staodaid encrypnon devices could be 
employed to mask the exaa namre of diese codes if one were concerned that would-be pirates would buy 
the iccognxtiott software merely to reverse engineer die universal embedded codes. The recognition 



SUBSTITUTE SHEET (RULE 26) 



wo 97/43736 



FCT/DS97/M351 - 



-33-^ 

software could simply uncocxypt the codes prior to applying die cea^nidon M^mwy^^ caagbc in dits 
disclosure. • 

The recogmdon software itself would certainly have a varies of &annes, bat the core task it 
would perform b demnmiing if dieie is some universal copyright code widiin i given image. The key 
5 questions become WHICH 16 of die total 50 universal codes it might comain, if any, and if there are 16 
found, what are their bit values. The key variables in determining die answen to diese quesiions are: 
registration, rotadon, magnification (scale), and exmnr. In the most general case widi no helpful hints 
whatsoever* all variables must be independently varied across all mumal combinaxitHis, and each of dse 50 
oniversaJ codes must dn be cbecked by adding and subtracting to aee if an enoropy decrease occurs. 

10 Stricdy speaking, dus is an enormous job, but many helpful hints will be found which make the job much 
sunpier» snch as havii^ an original image to compare to die suspeaed copy, or knowing du general 
orientation and extent of die image relative to an 9x10 prim paper, which dsn dmmgh single registration 
ftmfiTiir^tt^ can i^^ttmm all of die variables go some acrpptable degree* Hien it merely requires cycling 
duough die 50 univexsal codes to find any decrease in ennopy. If one does, dien 15 others should as welL 

15 A protocol needs to be set up whereby a given order of die 50 translates into a sequence of most 
significani bit tfaiough least significam bit of the ID code ward. Thus if we find dut universal code 
anmber *4* Is prescm, and we find its bit vahie to be "O", and chat universal codes T dirough •3* are 
definitely not present, then our most signifTcant bit of our K-bit ID code number is a "O". likewise, we 
find that die xxt lowest universal code present is number *7' and it rums out to be a dien our next 

20 most sigpific*"^ bit is a *r. Done prqieriy, this system can deanly trace back to die copyright owner so 
long as they registered dieir photogmphic paper stock serial number with some registry or with die 
mamifacnirer of the paper itself. That is, we look up in die registry dial a paper using univenal em bft d rfe d 
codes 4,7.1 U12a5,19*21,26.27,28,34,35,37,38.40. and 48. and having ±e embedded code 0110 0101 
0111 0100 belongs to Leonardo de Boticelli, an unknown wildlife pliotographer and glacier 

25 cinematographer whose address is in Northern Canada. We know diis because he dutifully registered his 
film and paper stock, a few miautcs of work when he bought the stock, which he plopped imo die "do 
postage necessary' enveiqie that the manufacturing oonqiany kindly provided Id make the process 
ridiculously simple. Somebody owes Leonardo a royalty check It would appear, and certainly die n^isciy 
has ^ffrrmatiKi ihis loyalty pigment process as part of its services. 

30 One final pomt is diat truty sopbistkated piiaies and odiers widi illicit imeuKHU can indeed 

employ a variety of cryptographic and not so cryptographic mediods to crack diese univenal codes, sell 
th^ and make software and hardware tools which can assist in die lemovii^ or distorting of codes. We 
i^^W aot ti'^rh fttw ii Bthnri g a$ pan of diis disclorore, however. In any event, diis is one of die prices 
which must be paid for die ease of universal codes and die applicadons diey open up. 

35 
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ynivgrsal Codes: 2) Un iversal Codes Baaed on Eteennirostic FonnuJas 

Tbe libcaries of umytisai codes requzn the storage and oansmitcal of Megabytes of independenit 
generally random data as the keys with which to ttoioclc the existence and idencny of sigoab and imagery 
uac have been nxariced widi universal codes. Alteniatiydy. various demninistic fommias can be used 
5 which 'generate*' what appear to be random data/image frames, thereby obviating dse need to store all of 
codes in memoiy and intenogate each and of the 'SO** universal codes* Deterministic fommias can 
also assist in speeding op the process of detcnnining the ID code once one is known to exist in a given 
signal or image. On the odier hand, detenninistlc fommias lend diemsdves to sJeuihing by less 
sophxsticaied pirates. And once sioithed, they lend themselves to easier communication, such as posting 
10 on the Interna to a hundred newsgroups. Hxeie may well be many applications which do not care about 
slenthing and publishing, and determinisdc formulas for generating the individual univexsal emfaeddftd 
codes migbr be just dse iii±ei. 

TT^g^al pndear 1> "Siflmle" Univefsai Codes 

15 This caiBgaiy is a bit of ahyfarid of die first two, and is most diiected ar miiy large scale 

is^emoDtaiions of ±e piindples of this technology. The applicaiicns employing this class are of the type 
where staunch security is much less importanr tban low cost, large scale implementation and the vastly 
larger economic benefis (hat this enables. One exemplary q^pQcadon is placement of identification 
iccognition units directly within modestly priced home audio and video instrumsntaiion (such as a TV). 

20 Such recognition nints would typkally mraxicor audio and/or video looimig for these oopyiighc identification 
codes, and titence triggering simple decisions based on the findings, such as disabling or enabling 
reconiing capabilities, or mcnanentrng program specific biUing meters which are transmitted back to a 
central audioMdeo service provider and placed onto monthly invoices. Lilcewise. it can be foreseen that 
"black boxes" in ban and other public places can monitor (listen with a microphone) for copyrighted 

25 T?af»»«aig and generate detailed r^xms, for use by ASCAP, BMI, and the like. 

A core principle of simple universal codes is that some basic indnstry standard 'noiseixke" aixt 
seamlessly rcpeoiivc patterns arc uijectcd mto signals, iixtages, and hnage sequences so that inexpensive 
recognition units can either A) decermine die mete existence of a copyright 'flag*, and B) additionally to 
A. dftw™^ ptedse ideniifiC3tio& mfdrmation wbicii ean fiicilitate more complex rt irni inn making and 

30 actions. 

In Older to hnpiement diis particttlai embodimeu. the basic principles of generating the indivkhial 
i TT tfvi d ^'^ noise signals need to be stmpiified in order to accommodaxe uopensive rccogmiion agnal 
processing drcoirry, while ^»i^^tafn^T1g the properties of effective randomness and holographic pcrmeatioQ. 
With large scale mdustry ado^Rion of these simple codes, the codes themselves would border on public 
35 domain information (mnch as cable scramblmg boxes are almost de &cto public domain), leaving the door 
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opea for deEezniixed pzrnies lo develop biack marta coumenneasurea, but ihis situation would be quite 
aimiogous to the scrambling of cable video and the objective economic analysis of stich illegal activi^. 

One prior an known to the applicant in this gecenl area of pro-acuve copytis^ detection is the 
Serial Copy Management System adopted by many finns in the audio industry. To the best of applicant's 

5 knowledge* ffaia system employs a ooo-audio "Sag" signal which is not part of the audio data stream, bat 
which is nevertheless grafted onto die audio stream and can indicaie whoher the associated audio data 
should or should not be duplicated. One problem with this system is that it is resthcted to media azxi 
instmnencatioo which can suppon this extra "flag" signal. Anodier deficiency is diat ±e flagging system 
canies no tdendty iufonnatian which would be useful in isaking more complex decisxcns. Yet anodxer 

10 difficulty IS that high quality audio sampling of an analog signal can come arbitiaiily dose to making a 
perfea digital copy of some (tigitai master and diere seems to be no provision for inhibiting this 
possibility. 

Appiicam's tedmology can be biought to bear on these and other problems, hi audio applications, 
Yideo, and all of the odier applicadons previously d iscuss e d. An exemplary application of simple univecsal 

15 codea is die fallowing. A sin^e industry standard "l.OOCXXW second of noise* would be dcfi^ 

most basic indicator of the presence or absence of die copyright marlnrg of any given audio signal. Fig. 9 
has an example of what the waveform of aa industry standard noise second might look like, both in die 
time domain 400 and die frequency domain 402- It b by definition a continuous ftmcnon and would adapt 
to any combination of sampling rxtrm and bit quantizations. It has a Doimallzed amplimde and can be 

20 scaled arbioaiily to any digital signal amplitude. TTie signal lerel and the first M*th derivatives of die 
signal are condnuous at die two boundaries 404 (Fig. 9C). such diat when it is repeated, die "break" in die 
signal would not be visible (as a waveform) or audible when played dirough a high end audio system. The 
choice of I seeond is arbittaxy in diis example, where the piecise length of die interval will be dexxvcd 
from considerations such as audibiUiy, quasi-white noise stams, seamless repeatability, sis^Ucity of 

25 rmgxudon pn)ccsaing. and speed widi which a copyright nmldng dctermu^^ The 
hijecdon of dus repeated noise signal onto a signal or image (again, at levds below human perception) 
would indicate die presence of copyright material. This is essenuaUy a one bit identification code, and die 
embedding of fnrthcr idcntificadon infonnation will be discussed later on in Uiis secticm. The use of diis 
identification technique can cxtemi far bcyrod die low cost borne implementations discussed here, where 

30 studios could use the technique, and monitcoing stations could be set up which linraUy monitor hundreds 
0 f cfaanneh of infbixnation sinraltaneously . searching for mariced data streams , and fiuthermore searching 
for die associated identity codes wfaidi could be tied in widi bilUng networics and royalty tracking systems. 

This basic, ytawd^i?^ noise stgoamre is seamlessly repeated over and over again and added to 
audio yg""'^ which are to be mailsd with die base copyright identification. Pan of die teason far die 

35 word "ample" is seen here: clcarty pirates will know about diis mdustty standard signal, but dieir Illicit 
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uses dozved fom this knowiedge, such as ensuxe or corruption, will be econooxically mimiscule relative 
to the econotoic vahie of the ovmil oecfamque to the nuss markei. For most high end audio this signal 
win be some 80 iq 100 dB down fsam foil scale, or even much further; each sttuacion can choose its own 
levds though cenaioly ihexe will be recommexidaiioas. The antplimde of the signal can be modiilarcd 

5 according to the aui^o signal levels to which die noise signanire is being applied, i«e. tbe anq>iititde can 
increase significantly wben a dram beats, but not so diaznaiicaUy as to become andible or objectionable. 
These msasuzes meztiy assist the lecognidon cucuitiy to be described. 

Recognition of the prraex2ce of this noise signature by low cost instrmnentation can be effecied in 
a variety of ways. One rests on basic modifications to die simple principles of audio sigoai power 

10 metering. Software recognition programs can also be written* and more sophisdcaiEd madtemadcal 
detecnon aigorirhms can be applied to andio hi order to malce higher confidence detecdon identifications. 
En such embodiments, detection of die copyright noise signature mvolves oompsrii^ the time averted 
power level of an audio signal widi die time sversged power levd of dm same audio signal which has had 
the noise signature subQacted from it. If the audio signal with tize noise signsture subtracied has a lower 

15 power level diat the imghaaged audio signal, then the copyri^ sigoatnre is presem and stnne stams flag to 
chat effect needs to be set. The main engineeiing subdedes hxvolved in maldng diis comparison include: 
Healing with sudio spccd playback discrepancies (eg. an instnmxcnt migiu be 0^% "slow* relative to 
exacdy one second intervals); and, ^^^li^g with the nnknown piiase of the one scond noise signature 
within any given audio (basically, us ""phase* can be snywiisre from 0 to 1 seconds). Anodier suteiety, 

20 not so central as Uie above two but which nonetheless should be addressed, is tbat die xecognidon circuits 
should not sidxract a higher iimp!?*^'^'^ of die noise signanire than was onginaUy embedded onto die audio 
signal. Foitnoatdy this can be accomplished by merely sid)ttacting only a small amplitude of the noise 
signal, and if die power level goes down* diis Is an mdication of 'hesdmg ooward a trough' m die power 
levds. Yetaoodierielatedsubdevisdiatdiepowerlevdchattges wiUfae very smaUrda^^ 

25 overall power levels, and ralCTkt iff m generally will need to be done with appropriate bit pitcisioo^ e.g. 32 
bh value operations and aocnmuLackms on 16-20 bit andio in die calculations of time averaged power 
levds. 

Gody, designing and packaging diis power level comparison processing circuitry for low coat 
applications is an engmeering opumizaiian task. One irade-off wHi be the accuracy of making an 

30 idotfification reJdlve to the "shoR-cucs" whidi can be msKie to die dtcnitiy m order to lower its ooa and 
complexi^. One embodiment for placmg dds recogmtlon circniny inside of ustrumeDtation is through a 
mngig programmable integrated circnit which is custooi made for the cask. Fig. 10 ftcm one such 
integrated dxont 506. Here die audio signal comes in. 500. dther as a d^ztal signal or es an analog 
CTgnai u> be digitized "^^^ die JC 5Q0. and die ouipnt is a flag 502 which is set to one level if the 

35 copyright noise signtfure is found, and to anodier level if it Is not found. Also depicted is die to diat the 
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standardized soise signamre waveform is stored in Read Only Memory, 504, inside d^ IC 506. Tlieie 
will be a sli^ dme delay between die application of a& audio signal to die IC 506 and die output of a 
valid flag 502, dne id the need to stonxior some finite portion of die audio before a lecognitloo can piace. 
In thi^ case, diere may need co be a "flag valid" output 208 wiiere die IC infonas the exteinal work! if it 
5 bas bad enough dme to make a proper decennioation of the presence or absence of die copyxigbt noise 

There are a wide variety of specific designs and philosophies of designs applied to accomplishing 
dse basic function of die IC 506 of Fig. 10. Audio engineers and digital signal processing engiceers are 
able CO generate several fimdamaualiy different designs. One such design is depicted in Fig. II by a 

10 process 599, which itself is sabjea to funher engineering optimiaanoa as will be discussed. Fig. II 
depicts a flow chart for any of: an analog signal processing network, a(figital signal processing neiworlc, 
or progzamming steps in a software program. We find an input signal 600 whidi along one patb is applied 
to a time averaged power meter 602, and the resulting power mitput itself treated as a signal P,,. To die 
upper ri g^g we find die standard noise signature 504 which will be read out at 125% of normal qml, 604, 

15 dins cfa^s^'ng its pia±, giving die •pimh changed noise signal" 606. Then the input signal has dns pitch 
changed noise signal subtracted in step 608. and this new signal is applied (o the same form of time 
averaged power meter as in 602, here labelled 610. The outpui of diis opcradon is also a time based 
signal here labelled as 610. Step 612 dicn subtraos die power signal 602 ftom the power signal 
610, giving an output difference signal ?^ 613. If die universal standard noise signature does indeed 

20 exist on die mput audio signal 600, dzen case 2, 616, will be created wherein a beat signal 61S of 
approaimateSy 4 second period will show up on die ouqiitt signad 613, and it remains to detea this beat 
signal wtdi a step such as in Fig. 12, 622. Case 1, 614, is a steady noisy signal which exhibits no periodic 
beadng. 125% at step 604 Is chosen arbitxarily bere^ where engineering considerations would doeimine 
an optimal value, leading to different beat signal frequencies 618, Whereas waiting 4 seconds in this 

25 example would be quite a while, especially is you would want to dcicrt at least two or three beats. Fig. 12 
outlines how the basic design of Fig. 1 1 could be repeated and operated iqmn various delayed versions of 
the input signal, delved by something like inMi of a seconds with 20 parallel circuits woridng in concert 
each OQ a »gmem of die audio delayed by 0.05 seconds from dieir neighbon. In diis way, a beat signal 
will ^ow tq) apptoximateiy every l/5di of a second and will look like a naveilmg wave down die columns 

30 of beat detection drcuhs. The esdstence or absence of diis traveling beat wave triggers die detection flag 
502. Meanwhile, diere would be an audio signal monitor 624 which would ensure diat, for exan^le, at 
least two seocButs of audio has been heard before setting the flag valid signal 508. 

Though die audio ezan^ile was described above, it should be clear to anyone in die an diat die 
same type of dcfinkion of some rcpetinve universal noee signal or image could be applied to die many 

35 other signals, images, pictures, and physical media already discussed. 
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The above case deais only with a singie bit plane of infonnatiotu i.e., the noise sigoanire signal is 
either there (1) or ir m*t (0). For many applications, it would be nice to detect serial number inibrmatioQ 
as well, wiuch couJd then be used for mm complex decisioDS, or for logging i&fbrmation on billtag 
siatement8orwliamot« The sanu principles as the above would appiy, but now diere would ^ 

5 independent noise signatnres as depicted in Fig. 9 insiead one single such signature. Typically, one such 
signature would be the master upon whidi the mere exigence of a copyright maildng is deeded, and this 
would have generally higher power Uum the others, and then the other k>wer power "Ldentificatioo'' noise 
signatures w<Kild be mbr^^ into audio. Recognition circuits, once having foood the exigence of die 
primary noise signanire, would then step through the other N noise signatures applying die same steps as 

10 described above. Wbeze a beat signal is detected, this indicates the bit value of r, and where no beat 
signal is deteoed, diis indtcatts a bit vahtt of '0'. It might be typical dsat H will equal 32, that way 2^ 
number of Tdmrtfir^" codes are available to any given industry employing diis technology. 

nf ih^ T^wilfl gy When the Length of die Identifieatiott Code is 1 

15 The principles described herein can obviously be ^lied in the case where only a single presence 

or absence of an identificaiion signal - a fingerprint if you will - is used to provide confidence thai sotnc 
signal or i™^'' is copyrighted. The example above of the industry standard coise signaoire is one case in 
point. We no longer have die added confidence of ihe coin flip analogy, we no longer have tracl±ig code 
capabilities or basic serial nmnber capabilities, but many ^plications may not require these attributes and 

20 die aifW fd simplicity of asingle fingerprint might ouiweigh dieae other attributes in any event. 

The 'WallpaDer' A' ^^'nffY 

The term "holografAic" has been used in this disclosure to describe how an identification code 
mimber is distributed ox a largely inicgial form throughout an encoded signal or image. This also refers to 

25 the idea that any given ftagment of die signal or image contains the entire unique idcniificanoD code 

oombw. As with physical hnplemeniaiions of holography, there are Umitations on how small a ftagment 
can become before one begms to lose diis property, where the resohidon Ihniis of the holographic media 
are the main fector in dds regard for holography itself. In die case of an uncomipted dtsnibuiion signal 
which has used the encoding device of Fig. 5, and which furdtermore has used our ^designed noise* of 

30 above wherein the zero's were randomly changed to a 1 or -1, then the extent of die firagment tetjuircd is 
merdy N contiguous sanqrfes in a signal or image raster line, where N is as defined previously behig die 
length of our identificaiion code number. This is an informational extreme; practical situations where 
noise and comipdoa are operative wUl require generally one, two of higher orders of magnirade more 
samples than diis simple nnmber N. Those skilled in die an will recognize diat there are many variabks 
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involved k pioniiig down precise statistics on die size of tbe smallest fragmeot with which an idendficatioa 
can be made. 

For mtofiai pmposes. die applicant also uses dn analogy that die unique idemification code 
mmdyer is *wailpapered" across and hnage (or signal). Tliat is, it is repeated over and over again ail 

5 throughout an image. This lepetitxoo of the ID code Qtmii)er can be regalar, as in die use of die encoder 
of Fig. 5, or lamiam itself, where the bits in die ID code 216 of Fig. 6 are not stepped througb m a 
normal repetitiye fia^on but radier are randomly selected on each sample, and die random selccdon stored 
along with die vahic of die output 228 itsdf. in any event, die infonnation carrier of die ID code, die 
individual embedded code signal, does change across die image or signal. Thus ai tbe wallpaper analogy 

10 summarizes: die ID code repeats itsdf over and over, but die patterns diat each repetition imprints change 
randomly accordingly to a generally unsleutfaable key. 

Lp^syDauComptessjon 

As earlier mentioned, applicant's prefcned forms of idemificatiQn coding widisia&d lossy data 
15 compressian, and subsequent decompression. Sudi compression is flndnxg increasing use, paitlcnlariy in 

comexts such as die mass distiibutioQ of digitized eneercainment programming (movies, etc.>. 

While data encoded according lo die disclosed tedmiqaes can withstand all types of lossy 

compression known lo applicant, those expected to be most cammerdaity important are tiie CCTTT G3. 

CCnr G4, JPEG, MPEG and JBIG compression/decompression standards. The CCITT standards are 
20 widely used in bladt-and-wbiie document compression (e.g. facsimile and document-storage). JPEG is 

most widely used widi still hnagcs. MPEG is most widely nsed widi moving images. JBIG is a likely 

snccessor id the CCTTT standards for use with black-and-white imagery. Such techiuquBS are well known 

to diose in die lossy data compressioQ field; a good overview can be found in Pennebaker et al, /PEG, Sciil 

Imag€ Data Comprtsaan Standard^ Van Nostraod Rehdiotd, Y.» 1993. 

25 

Towards Steganogranhv Proper and die Use of diis Technology in P assing More Complex Messages or 
Information 

This disclosnie conccntraics on what above was called wailpapcnng a smgie id rntific a tio n code 
across an entire signal. This appean to beadesiiable fcaniie for many applicarions. However, diere aie 
30 odier applicadoas where it might be desirable to pass messages or to embed very loog strings of penmem 
ideniiikation infotmatioa in signab and mages. One of maay ^ possible applications would be where a 
given signal or image is meant to be ^anqiniatwi by several different groups, and diat certain regions of an 
image are reserved for each group's identificatkm and iosenion of pertinent manipulatitm informaiioi. 

b diese cases, die code word 216 in Fig. 6 can acnially change in some pre-defined manner as a 
35 ftmrnrm of slgoal oT image posidoo. For example, m an image, -die code could change for each and every 
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rascer line of ibe digital image. It might be a L6 btc code word, 216. bat each scan line would bave a new 
code word, and dus a 480 scan line image could pass a 9^ (^^ A receiver of 

die message would need to bave access to either the soise signal stored in locmoty 214. or would have to 
loiaw the universai code stnicnue of die noise aides if diat mediad of ooding was bemg used. To die best 
5 of appiicaot's knowlec^, this is a novel ^iproach to the nature field of si^aDography. 

b all duee of die foregoing ai^licaiioas of umvenal oodes^ it will often be desiiable to append a 
shon (periiaps or i6*bit) private code, whxdi users would keep in dicir own secured places, in additira 
to die univeisal code. Hus affoids the user a further modicom of security s^ainsc potential erasure of the 
universal codes by sophisticated pirates. 

10 

ft pptkant's Prior A pplicatiop 

The Detailed Descripciott to this point has simply repeated the diieiosuie of ^iicam's prior 
international ^pUcoion, laid open as PCT publicatioa WO 95/14289. Et was reproduced above simpty to 
provide context kx die disdosnie which fiollovs. 

15 

Que Master Code Signal As A Distinction From N Independem Embedded Code SignaJs 

In certain sccaoas of diis disclosare, pcihaps exemplified in die section on the realmne encoder, 
an coniomizing stqi was taken whereby die N indqreodcnt and sourcc-signal-cocxtcnsiye endaeddcd code 
signals were so designed that ihe non-zBro elcnjHUs of any given embedded code signal were tmiqae to just 

20 diat embedded code signal and no others. Said more catefully, certain pixds/sample points of a given 
signal were 'assigoed* to some pre-detexmised m'th bit locaticn in our N-bit identification word. 
Funheimaie^ and as aDodter basic opdmization of impiemestation. dae aggieg^ of dsese assigned 
pixelsysamples across all N embedded code signals is precisely die extent of the source signal, meaning 
each and cveiy pittUsample location in a source signal is assignfd one and only one m'tb bit place in wir 

25 N-bii identi&aiion wonL (TTris is not to say. however, diat each and every pixel MUST be modiSed). 
As a matter of simtriificatioa we can dicn talk about a single master code signal (or "Snowy image') laiher 
than K mf1*pfM*^ signals, lealiziDg that pceHlefined locadons in diis masrrr signal correspond to unique 
bit locations in our N-bit identification word. We Ihercfbre construct, via diis drcoiions lomc, this rather 
ample CT U ffrp * on the smgle master noise agoal. Beyond mere eeonondzation and simplification, dne axe 

30 also performance xasam for diis dufu pcimatily derived from die idea diat indxvidoal bit places in our 
N^iitiitaaitoiiott word are no longer "mmpeting'' for die infbimadon carrying capacity of a single 

pixel/sample. 

Widi diis single master more clearly understood, we can gain new tosigbts uMo other sections of 
diis disdosuie and explore fimher details widiin the given ^plication areas. 
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Mnre of Derennmhtic Universal Codes Using t he Master Code Concepr 

One case in point is to ftnther explore die use of Detetministic Uoivctsal Codes, labelled as item *2" in 
die secxiOQS devoted to univeisal codes. A given user of this tedinolosy may opt far the fbllowii^ variant 
use of die prindples of tliis technology. Tlie user in ([oestion migtn be & inass distribmor of tiom^ 
5 but dearly die principies would extend to all other potential usera of diis technology. Hg. 13 pictoxlalLy 
represents die steps involved. In die exampie die user b one * Alien Producdons. ' They first czeace an 
image canvas which is coexiensivs lo die size of the video frames of their movie *Bud's Advennues." On 
diis canvas they pxint d» oaoe of the movie, they place dieir logo and company oame. Furthomoze, diey 
have specific information at the bottom* such as the distribudon lot for die mass copyic^ diat diey are 

10 currently crankmg out, and as mdicated, they acuially have a unique ^ame aumber hidxcaced. Thus we 
find die example of a standard image 700 which forms dxe initial basis for die creation of a master Snowy 
image (msscer code signal) wUch will be added mto die original movie fiame. ctcatmg an ouqrat 
disoibut^le {came. This hnage 700 can be eidier blact & white or color* The process of mnung diis 
image 700 into a pseudo random master code signal is ailnded to by the eocrypdon/scsambUng routine 702, 

15 wherein die onginai image 700 is passed duough ax^ of dcrais of well known scrambling mediods. The 
depictiOtt of the amnbcr "28' alludes m die idea thai dicre can actually be a library of saambling methods* 
and die particular mediod used for this particular movie, or even for diis pardcular frame, can change. 
The result is our claaic maaer cede signal or Snowy Image. In general, its brighmess vahies »e large 
and it would kx* very much like die snowy image on a television set tuned to a blank channel, but deariy 

20 it has been derived from an uiformaiive image 700, transformed duough a scrambUng 702. (Kote: die 
splotchiness ci die example picmre is acaially a radier poor depiaion; it was a fanctiott of the crude tools 

available to the investor). 

This Master Snowy Image 704 is dien the signal which is modulated by our N-bit identificadon 
woxd as oudised in odier secdons of die disclosure, die resulting modulated signal is dien scaled down hi 
25 brighmess to die acceptable pexceived noise level, and dsea added to die origisal frame to produce die 
distributable frame* 

There aie a variety of advantages and feamrcs thai die mediod depicted m 13 affords. There 
are also variations of dicme whhin diis overall variation. Qcariy, one advantage is diat users can now use 
more hmiitive and peisanaliied mediods for stamping and signing d^ Provided tfiat die 
30 eacrypcion/sciamhluig routines, 702, arc indeed of a high secoriy and not published or leaked, dacn even 
ifawoaid-be prate has hnwrtedge of the togo image TO), they should not be ab^^ 
be able to slcudi dm Mater SiMrR7 Image 704, and dius diey sh^ 

were. On the other h^"**. sioqile cncryptiDn inunnes 702 may open die door for cracking the system. 
Anodier clear advantage of die mediod of Fig. 13 is die ability to place ftahcr hifonnation mio die overaU 
35 protective process. Stricdy spcakmg, die information contained in dm logo hnage 700 is not di^ 
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earned ia die faial dlsvilnuable frame. Said another way, and provided diat dte eocryptioa/jcraDbling 
routine 702 has a siraightfoiward and kstmi deoypcioo/dcsczambling method whidi toleraies bit mmcaiioa 
errors. It is generally tmpossibie co My re«crexe die image 700 based upon having the diathbutabie 
frame, the N-^ic idemiflcatiott code word, the brightness scalmg facnr used, and die number of die 
S dectyptmn rsutme to be osed. The reason diac an exact recreation of die image 700 is impossible is due to 
the scaling operation itself and die concomitant bit truncation. For die present discossioa^ this whole issue 
is somewhat academic* however. 

A variadoo on die theme of Bg. 13 is to acoially place die N4)it jdauificaixim code word diiocily 
into die logo image 700. In some xnsc tiiis would he seJf-re&rezmal. Thm wben we puH out our noted 
10 logo image TOO it already contains visually what our identification word is^ dien we apply encryption 
zoutine #28 co dns image, scale it down, dien use diis version to decode a suspect image using die 
ttchniquesi of dtis diaclosmc. The N bit won! thus found should loatch die one contained in our logo 
image 700. 

One desirable feature of Che encryptian/scramblhig routine 702 might be (but is ceitainly not 
15 required co be) that even giva a small change in the input image 700, such as asingie digit change of die 

frame number* there would be a huge visual change in die onrpui scrambled master snowy image 704. 

Likewise^ the actual scrambling rotmnc may change as a foncticn of frame nmnbers, or certain *seed" 

tmmbers typically used within pseudo-randomizing toctioss could change as a ftmcnon of firame number. 

All manner of vaxiadons are Uius possible, all helping to maintain high levels of security. Eventually, 
20 engineering o ptimizati on consictenmons will begin to investigate the relationship betweea some of these 

randoinizing methods* and bow they all relate to nudmaunng acceptable signal ssrengdi levels thiough die 

process of tiansfbtnung an uncozopressed video si i ri"! Into a compressed video stream such as with the 

MPEG compression methodologies* 

Anoitter desired feamre of the encryption process 702 is that it should be informadonaliy efficient, 
25 i.e.. that given any random inpnt, it should be able to outpm an essentially spatially unif<Hm noby image 

with linle to no lesidnai spatial paucns beyond pure taodonmess. Any residual ctnrelaced paicems vhll 

CQCtribute to mefSctency of encoding the N-bit jdwitiftcatinn word, as well as opening up further tools to 

would-be pirates to break die system. 

Another featme of die rvh^ of Ing. 13 is that there is more innutiooal appeal to using 
30 recognizable symbols as pan of a decoding system, whidi should dieo translate favorably m the essendally 

1^ eaviromnem of a coottroom* It strengthens the simplicity of the coin fiip vernacular mentioned 

elsei»iiere. Jury meirdiers or judges will better relate to an owner's logo as being a pteoe of die key of 

noogttizing a suspoci copy as being a knock-off* 

It should also be mentioned t^a^^ strlcdy speaking, the logo image 700 does not need to be 
35 randomized. The steps could equally apply straight to die logo image 700 directly. It is not entirely clear 
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co Che inveuor wha pnctknl foal this migb have. A crivial exceosioa of tfads concept to the case wheie 
Nsl is wfaeie, simpiy and easily, the logo image 700 is meidy added to an ongmal hn;^ at a veiy iow 
brighmess level. The iaveocor does not piesuiz» this trlviai caae co be at ail a novelty. In many ^iis^s cbis 
is sunilar to die ^e old issue of subliminal advectismg, wheie the low fight level panenis added to an 
5 image arc iecogni22faIc to the human eye/brain system and - supposedQy - operating on the human brain at 
an ancooscions level. By pointing otu these trivial extensions of the current technology, hqieluily there 
can arise fuxtber dazity which distxaguishes appiicam's novel principles m relation to such well known 
prior an techniques. 

10 

^Jxri Ahrirfiyri Alpbanomeric Code Sets and Odien 

It is desirable hi some applicatiODs far the N-bit ide n t ifi ca ti on word to acmally aignify names, 
companks, strange words, messages, and die like. Most of this disclosure focuses on using die N-bit 
ideotificaiion woid merely for hi^ srailirtcal secmiiy, mriffied tracking codes, and other index based 

15 message carrying. Tlie mformarton carrying capacity of 'invisible signamrra* inside imagery and audio is 
somewhat however, and dnis it would be wise tD use our N bits efficiently if we acmally warn to 

"speE out' alphamunehc items in the N-bit ideniificadoii word. 

One way id do this is to define, or to use an already existing, rednced bit (e.g. less than S*bit 
ASCII) 5tandardi39Bd codes for passing alphamimeric messages. Hits can help to satxsiy dus need on the 

20 part of some applications. For example, a simple alphannniexic code could be built on a S-bxt mdex table, 
where for eaompie die letters VXQ, and Z are not inchided, but die digits 0 dirougb 9 are hichided. In 
this way» a 100 bit identification word could cany wit!i It 20 alpbanumenc symbols. Another alternative is 
to use variable bit lengdi codes suet as the ones used m text eompresstoo routines (e.g. Huf&nan) whereby 
more frequently used symbols have shorter bit tengdi codes and less firciniently used symbols have longer 

25 bit lengdu. 

More on Detecting and tti»rf>fpii^^^ ( y the N-bit Identification Word in Sust»CT Signals 

Classicaliy s^iealdng, die detection of die N*bLt idmriflratirm word fits niody hxto the old art of 
detecting known signals in noise. Noise in diis last statement can be intetpreted veiy broadly^ ^ven to the 
30 point wfaexe an image or audio track itself can be considered noise, relative to die need to detect die 
underi3ring signamre signals. One of many references to dus older an is the book Kassam, Saleem A., 
"Signal Detecdon m N«i*Gaussian Moise/ Springer-Verlag, 1988 (generally available at weU stocked 
libraries* e.g. available at die U.S, Library of Congress by catalog number TK510Z.5 .K357 1988, 

To the best of this inventor's c ur r ent understandii^, none of the material 
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ia this book is dizccxiy applicable to the issue of discovering ±c polarity of applicant's embedded signals, 
but die broader priocipies aie indeed appUcabie. 

Da particnlar, secdon l«2 'Basic Ccmcepts of Hypotbesis Testing" of Kassam^s book lays out the 
basic concept of a binaty hypodxesia, assigning die value T to one bypodiesis and die vaine *0' to die 

5 otiier hypodtesis. Tlie last paiagi^ of chat section is also on poim regardbig die eacli^^ 

embodiment, 1.6.. dial the '0* hypothesis corresponds to "noise only* case« whexeas die *!* conespoods 
to die piesence of a signal in the observations. Applicant's use of true polaiiiy is am like diis, however, 
where now die "Q" corresponds to die presence of an inverted signal radier dian to 'noise-only.' Also in 
the present asAmdimfint, die case of "noise-only " is e&ectively ignored, and diat an identification process 

10 will eidier come up widi our N-bit tdenrification word or it will come up widi 'garbage.' 

The continned and inevitable engineeticg improveoient in the detection of embedded code signals 
will ondoubtedly botxow heavily ftom this generic field of known signal detection. A conmum and 
well-known cechnuiue in diis field is die so-called 'macfaed filter," which is pici rienta lly diacnssed eady in 
settkm2of theKassambook. Many basic texts on sigtial processing ixsdude discussions cm this niethod 

15 signal deiecdoo. This is also known in some fields as cotxelation dececdon. Fuithennoie» when die phase 
or locactott of a known signal is known a priori* such as is ofiten the case m applicacions of this technology, 
dien die matched filter can often be reduced to a simple vector dot prodtia between a suapea image and 
die rmbr^^^ associated widi an m'di bit plane in our N-bit idendficanon word. This dien 
rqucsntts yet anorficr simple "ddocticni aigorilhm" for taking a suspea unage and producing a sequence of 

20 Is and with die intention of detetsnmng if diat series corresponds u> a pie-e m bedded N-bit idendficacion 
word, bi words, and widi reference to Fig, 3, we nm dirougb die process ateps ap dirough and mehidhig 
the snburaaing of die odginai image finm the suspect, but die next step is meicly to step duough aU N 
random in dftwyj*^"* signals and pertem a simple vector doc prodna between these signals and the 
diffmnce signalr and if diat dot product is negative, asagn a '0* and if diat dot ptodnci is positive, assign 

25 a'l.' C^fid flB*'y™ fl^dri^ 'one of many" algprithms will show its similariry to the tradidonal matched 
filter. 

Thoe arc also some 'nnm^vnt» nnpitivCTiems to the -matched filler* and "ojirclation-cype" diat 
can provide ' ^ ^ m'^ ability to properly detect very low level embedded code signals. Some of diese 
improvements are derived &am principles set forth io the Kassam book« odiers are generated by die 

30 mvemor and die inventor has no knowledge of dieir being developed in odier papers or worics. but neidier 
bas die hivemor done fidly extensive searching for advanced signal detecdcn ledmitpxes. One such 
teefam^pm is pedi^ best exemplified figure 3.5 m Kassam on page 79, whextiii die« are certain plots 
of die varioua locally cptimmn weighting coefficients whidi can apply to a general dotisrodnct algoridsntc 
approadi to (kiection. In other words, rather than pcrfonmng a simple dot prodna, each ciemcatal 

35 multqjlicadan operation m an overall dot produa can be weighted based upon known a priori statisdcal 
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in^ramdon about t&e diffexence signal iisdf, i.e., [he signal mxhm wtich the lo\i^ kvel ksowa signais arc 
being soughi. The interesied reader wbo is not already familiar witb these topics is eooouraged to read 
ttaptcv 3 of Kusam to gain a fuller understanding. 

One principle which did noi seem K) be explicitly present in the Kassam book and which was 
5 developed ru^memanly by the tnvenior involves the eitploitation of the magnitudes of the staiiaticai 
properties of the known signal being sought relative to the tsagntiude of the statisticai properties of die 
suspect stgnai as a whole, b paidcular, ±e pzobleznanc case seems lo be where the embedded signais we 
are looking fbr are of mnch lower level than the noise and corruption present on a difference signal. Fig. 
14 annmpts to set the stage fbr the reasoning behind this approach. The top figure 720 contains a generic 

10 look at the differences in die histograms between a typical 'problematic' difference signal, i.e., a 

difference signal which has a much higher overall energy than the embedded signals that may or may not 
be widiin k. The term 'mean'Temoved' simply means that the means of botU the difSerence signal and the 
embedded code signal have been removed, a cooomn operation prior to per^Dnning a normalized dot 
product. The lower figure 722 then has a generally similar histogram plot of die derxvativea of die two 

15 signais, or in die case of an image» be scalar gradients. From pore inspecdoc it can be seen thai a simple 
ttuesholding operation in die derivative transform domain^ with a subsequent conversion back into the 
signal domain, will go a long way toward removing certain innate biases on the dot product 'rccogniiion 
algorithm' of a few paragraj^ back. Tlresholding here refers to ±c idea that if the absolute value of a 
difference signal derivaiive value exceeds some threshold, dien it is replaced singly by that dircshold 

20 value. Hic diicshold value can be so chosen to contain most of die histogram of the embedded signal. 
' Anodicr operation Which can be of minor assistance in 'aBeviating" soTC 
dot product ;dgoiitfam is the removal of the low order fccquencxes in Uw diSiacncc signal, i.c.. nmnmg die 
difference signal duongh a high pass filter, where the cutoff frequency for die high pass filter is rcladvely 
near the origin (or DC) frequency. 

25 

rftiwiderations for RecQ gniTi rfl ; F.mhgdded Codes on Si^js Which Have Been Compressed and 
Decogmressed. ftf Altemativehf. for ReeooniT m^ Embedded Codes Within Any Signal Which Ha^^ 
Tlndergone Some Known Process Which Creates N on>Unifi>nn Error Sources 

Long title for a basic concept. Some signal processing opciarions. such as compressing and 

30 decompressing an image, as widi die JPEG/MPEG formats of image/video compression, cicaie erron in 
some given ffansfoimdomam which have certain coxxciatiaosa^ Using JPEG as an example, 

if a given image is ctnnpicsscd dien decompressed at some high compicssion ratio, and that lesuWng 
i ffpy Is iben fourier transfonned and comiiaied to the fomier transform of die original uncompressed 
in^e. a definite pattern is clearly visible. Hiis patterning is mdicarive of correlated oror. i.e, error 

35 which can be to some extent quantified and predicted. Tbe prediction of die grosser properties of diis 
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comlated enor can chea be used to advantage in the hereiofoie-discussed methods of reGognizing ibe 
embedded code signals vithin some a^ect image which may have ondergone either JPEG compression or 
any other operation which leaves these telltale correlated error signatCTCS. The basic idea is that in areas 
where there are ioiown higher levels of error, the value of the lecognidon methods is diminished relative to 

5 dse areas wtdi known lower levels of eorxeiaied errors. It is often possAIe to quantify the expected levels 
of ecror and use this qaamificatian vo appxopiiateiy weight the retransfoxmed signal values. Usmg JPEG 
compression again as an exan^le, a suspect signal can be fouiier traosfomied, and the foorier space 
represeaiation way ciearfy show the telltale box grid pattern. The fourier space signal can then be 
"spatially filtexed' near the grid points* and chis filtered cepresentatitm can then be traDsformed back mto 

10 its regular h'tp or space tioinain to then be run ihnnigh the tecogniiion method^ presented in this 
disdosore. Liltewiser any signal processing mohod which creates oon-uniform error sources can be 
tran sforme d into the domain m which these error sources are ttoxHmifotm, the values at die hi^ pomis of 
the error sources can be anexuiaied, and the thusly ^filtered" signal can be transformed back into the 
time/space domain for standard recognition. Often this whole process will include the lengthy and arduous 

15 step of 'charactetizii^* the typical correlated error behavior in order to "design" the appropriate filtering 
profiles. 

■jCTHMATURE CODES" atld "INVTSIBLE SIGNATURES* 

Briefly and for the sake of clarity^ the phrases and terms "signatures,' "invisible signatu res/ and 
20 'signature codes" have been and will continue to be used to refer to the general techniques of this 

technology and often refer spediically to the composite embedded code signal as defined early on in this 
disclosure. 

MfiRF nFTAITj; QN EMBED DING SIGNATURE CODES INTO MOTTON PICTURES 
25 Just as there Is adisiincdan made between the JPEG standards for compressing stili images and 

the M?EG standards for compressed motion images, so too should there be cfisdnctions made boween 
placing invisible signatures into still images and placing signamres mm motion images. As with the 
JPEG/MPEG distmcdon, it is mx a mat ter of different foundations, it is the fact diat with motion images a 
new dimension of eogmeeiing opthnization opens up by the inclusion of time as a paiazncter. Any 
30 textbook dealing iwdi MPEG will surely contain a section on how MPEG is (generally) not merdy 

applying JPEG on a frame by frame basis. It wiH be the same widi the application of the princtples of dus 
technology: generally speaking, the placement of invisible signatures nno nmtion h n age seqne c ccs will not 
be tjmpi y hidepemlently placing faivisible slgnanxrcs huo one frame alter die next. A variety of time-based 
considerations come mto play, some dealing with die psychophysics of modon image perceptim* others 
35 driven by snnple eosi engsiming consideraticms. 
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One embodimesi acniaUy uses (he MPEG compressioa sumdaid as a piece of a solutioa. Other 
moaoa image compressioo could equally well be used, be diey already isvented or yet to be 

invemed. This exam{4e also utilizes die scrambled logo image appcoach to geuexaiiiig die master snowy 
image as depicted ia Fig. 13 aod discussed ta the disdosun. 
S A 'compressed master snowy image* is independcntiy rendered as depicted in Fig. 15. 

"Rodered* icfen to the generally well known technique in video, niovie and auimaiioa production 
whereby an image or sequence of images is created by consouctive techniques such as computer 
iustrucdGns or the drawing of animation cdls by hand. Thus, "to render' a signature movie in this 
example is es^dally to let eidier a compmer czcate it as a digital file or co design some custom digital 

10 eiectroiuc circuitry to create iL 

The ovefalt goal of die procedure oudined izi Hg. IS is to apply the invisible signatures to die 
original movie 762 in such a way that the sigtiaQxtes do not degrade the connnerdal value of die movie, 
memoriatized by die side-by-side viewing, 768, AND ia such a way diat the signature optimaUy survives 
diroogh die MPEG compiesaian and deooa^tessioa process. As noted eaxiier* du use of the MPEG 

13 process in particular is an exanqiie of die generic process of compression. Also it should be noted that die 
oample presented here has definite room for engineering vaxianons. In particular, diose practiced in die 
art of mixion picture compression will appreciate the feet if we stan out wiUi two video saeams A and B, 
and we compress A and B separately and combine dicir results, then die resultant video stream C will not 
generally be the as if we pro-added the video streams A and fi and compressed this restdtant. Thus 

20 we have in gaierai, e.g.: 

MPEG(A) + MPEG(B) =\= MPEG<A+B) 

where »\« is not equal to. Ilufl is somewhat an abstract nodon to introdiKe at dus poim in the disclosure 
25 and win become more dear as Fig. 15 is discussed- The general idea, however, ia diat diere will be a 
variety of algdnas that can be used to optimize die pass-dmragh of •invisible" signamcs dirou^ 
compiBssion procwhnes. Qearly, die sanM principles as depicted in Fig. 13 also work on still images and 
tiie JPEG or any other still Image conqncssion standard. 

Turning now to die daails of Fig. 15, we begin widi die simple stepping tiirough of all Z frames 
30 of a movie or video. For a two hour movie played at 30 frames per second, 2 uitDS out m be 

(30*2*60*60) or 216.000, Tie inner loop of 700, 702 and 704 merely mimics Fig. 13 's steps. The logo 
frame opdonally can change dniing ±c stuping ffaraugh frames. The two arrows emanadng from die box 
704 represent bodi die condmudon of die loop 750 and die depositing of output frames into die tendered 
master Snowy Image 7S2. 
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To uke a brief but potentially appropriate digression at ihis point, tiie ase of ±e concept of a 
Markov process brings certain clarity to die discussion of optimiziag the engineering izoptetnencatton of the 
methods of Fig. IS. Briefly, a Madtov process u one in whidi a sequeoix of events la^ 
general thcit is no meaoty between oue step in the sequence and the nsxt« In the context of Fig. 15 and a 

3 sequence of images, a Marfcovtan sequence of images would be one in which there is oo apparent or 
jpp rfn' ahtff corxeiadon between a given frame and the noL Imagine taking die set of all movies ever 
produced, stepping one frame at a time and selecting a random ftame &om a random movie to be insoted 
intD an output movie, and diea stepping through^ say, rate minmc or 1800 of these frames. The resulting 
'movie" would be a fine example of a Maricovian movie. One point of diis discossion is that depending on 

10 how the logo frzms& are reodered and depending an how the ennyptioayscrambling seep 702 is perfbnned, 
±e Master Snowy Movie 752 will exhibit some generally quamifiabie degree of Markovian characteristics. 
The point of diia point is that die conqircasioa piocedutc itself will be afEbcted by diia degree of Markovian 
nature md duis needs to be accounted for in designing die process of Ftg. 15. Likewise, and only in 
general, even if a fally Markovian movie is created in the ffigh Brightness Master Snowy Movie, 752. 

15 diea d» processing of compressing atid decompressing that movie 752, represmted as die MPEG box 754» 
will break down some of the Markovian nature of 752 and create at lease a marginaUy non-Markovian 
compressed master Snowy Movie 756. This point will be utiiircd when the disclosure briefly discusses die 
idea of usmg nmltipie frames of a video stream in order to find a single N-bit idemificatkHi word, dxai is, 
the r»™ N-bit identification word may be rmhPflfled into several frames of a movie, and k is quite 

20 reasonable to use die infoimation doived fcom diose multipie frames to find that single N-bit identxficadon 
word. Hie non-Maxkovian nanire of 756 dms adds certain tools to reading and recognizing die invisible 
signatures. Enough of this m^mt 

With die intent of pre-conditioning ite ultimately utilized Masicr Snowy Movie 756, we now send 
die rendered High Brighmcas Master Snowy Movie 752 through bodx the MPEG oon^rression AND 

25 decompression procednre 754. With die caveat ptcviousiy discussed where it is acknowledged that tii^ 
MPEG compression process is generally not distributive, die idea of the step 754 is to crudely segregate 
die initially rendered Snowy Movie 752 into two components, die component which survives die 
con^iression process 754 which is 756. and die oomponent wluch does not survive, also crudely csiimamd 
using die dififerencc opcradon 758 to produce die "Cheap Masicr Snowy Movie' 760. The reason use is 

30 oade-of die ddlbesately loose term 'Cheap' is diat wc can later add diis signanne signal as weU to a 
distrilwtable movie, knowing tiiar it probably won't survive common oompression processes but diat 
Dcvcrdidess it can prtjvide 'cheap" extra signamre signal energy for applications or situations whidi will 
never cjqjeiieacc compression. [Thus it is at least noied in Fig. 15]. Back to Fig. 15 proper, we now 
have a rough cut ai signamrcs which wc know have a higher likelihood of airviving intsa dirough die 

35 compression process, and we use this 'Compressed Master Snowy Movie* 756 to dim go dirough this 



SUBSHTUre sheet (rule 26) 



wo 97/43736 



FCT/USJ7/08351 . 



.49.. 



ptocednre of being scaled down 764« added o) the originai movie 766, producing a candidate disnibutable 
movie 770, tbea compared to tbe odginal movie (769) to eosuie chat it loeets whatever commerczally 
viable criteria which bsm beea set 19 (i.e. ±e acceptable perceived Doise level). The anew iioiQ the 
side-hy-side step 763 hack m the scale down step 764 ca i respo a ds quite directly to die 'experiment 

5 visually...' step of Fig- 2* and the gain control 226 of Fig. 6. Hiose practiced in die art of nnage and 
aiiHtfi information dteoiy can recognize (hat die whole ofBg. IS can be siunmanzed as atten^g to 
pre-condition die invisible signature signals in soch a way diat ibey are better able to wi±staxid even quire 
aj^wedable compression. To reiteraie a previously mcniioned item as weU, diis idea equally applies to 
ANY such pre-identifiable process 10 which an image* and image sequaice, or audio track might be 

10 sublccted. Hiis clearly iociudes ±e JPEG process 00 still images. 

A AiitiDnal Elements of the Realtim e Encoder Chruirrv 

It shouhi be ooted dtac die method steps repieseated in Fig. 15, generally following &om box 7S0 
op throng die creation of die con^sacd master snowy movie 756, could widi ccnaia modification be 
15 \f ^ h^»ntMi m hardware. In panicniar, die ovaalt analog noise source 206 in Fig. 6 could be replaced 
by such a hardware circuit. Ukewise the steps and assodated procedures depicted ui Bg. 13 couW 
impiii^tTiMirgd 20 hardware and replace die aralog noise source 206. 

Recognition based on more than one frgmg: non-M arkovijn signanncs 

20 As noted m the digression on Markov ami noa-Marfcov sequences of fanages, it is pointed om once 

again diat m such dicumsiances where die embedded invisible signature signals arc non-Markovian hx 
nature, i.e., that dicrc is some correlation between tba master snowy image of one frame w dial of the 
next, AND furdiennore diat a single N-bit idcntificadon word is used across a range of frames and diat die 
sequence of N-bit ideitfificatioo words associated with tbe sequetjce of frames is cat Marfcoviaa in nauire, 

25 dicn it is possible to utilize die data from several frames of a movie or video in order to recognize a single 
N-bit idenrifeation word. All of diis is a fancy way of saying diat die process of recogmsang die invisible 
signanncs should use as much infoimaiian as is available, in diis case transiting to multiple frames of a 
motkm image sequence. 

30 HPADTO VTOTFTCAT10K 

The concept of die •header' on a cfigiial image or aucfio Sic is a wcil established piacdce in die 
an. ThciopofRg. 16 has a simplified took at dw concept of die header, wherein a data fflc begi^ 
generally a comprehensive set of information about die fflc as a whole, often mduding information about 
Who die audior or copyright holder of die data is Jf dicrc is a copyright holder at aU . Tins header 800 is 
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then typically followed by the data itsdf 802, such as aa audio stream, a digital iznage« a video streasu or 
compiesssd versions of aay of these iiexns. Thi^ is all exceedingly known and common in the indusay. 

Ooe way in which the prindples of chls technology can be employed in the seivice of information 
imegricy b generxcsdly depicted in ±e lower diagram or Fig. 16. In general, the N-bit idemi&ation word 

5 can be used to essentially 'waQpaper^ a given simple message dirougbout an image (as dq»icted) or audio 
data stream, thereby reinforcing some message already contained hi a traditional header. This is refined 
to as *header vetification" in the title of this secdon. llie thinldng here is that less sophisticaied would-be 
pirates and abnscn can alter the infonnation content of header infonnatiaau and the more secure techniques 
of this technology can thus be used as checks on the verzcity of header inforxnadan. Provided that the 

10 code message^ such as "joe^s image" in the header, matches (he repeated oxessage diFoughout an image, 
thai a aser obtaining the image can have some higher degree of confidence diat no alteration of the header 
has taken place. 

Likewise, the header can actually cany die K-bit ideniificadon word so that ffae fact that a grven 
data set has been coded via the mediods of tfiis technology can be highlighted and the verificatioa code 
15 built light into die header. NamraUy, this dam fUe format has not been created yet since me principles of 
this tedmoiogy axe currently not being employed. 

TWE 'BOPreR'; THE ABILITY TO LARGgLY REPLAC E A HEADER 

Although all of the possible applications of dje following aspect of applicant's technology are cot 
20 fiiUy developei it is ocvctthclcss presented as a design alternative dsai may be important some day. The 
title of diis section contains the silly phrase used to describe diis possibility: die "bodier." 

Whereas the previous section outlhied how the N-bU identification word could "verify* 
information coutainBd within the header He, there is also die prospect that diese meUiods could 

cnmpletdy r^lacc die very concept of the header and place the information which is traditionally stored m 
25 the header directly hun die digital signal and empirical data itself. 

This could be as simple as staixiardizing on, purely for example, a 96-bit (12 bytes) leader suing 
on an odicrwisc cntiieiy empirical data stream. This leader string would plain and simple contain die 
numeric length, in cicmcatal data units, of the entire dau file not inchiding the leader string, and the 
number of bits of depth of a single dai3 dcmcm (eg. its nmabcr of grey levels or die number of discrete 
30 signal levels of an audio signal). From diere, miivcrsal cedes as described m this ^edficadon would be 
used to read the N^iitident{fiCBiionw<ml written dirocdy within ti^ The leogtb of die 

empirical data would need to be long enou^ to concain the fidl N bits. The N-bit word would effecuvely 
transmit what would otherwise be contained in a traditional header. 

F?g, 17 depicts such a data fbrmai and calls it die "universal empirical dau format." The leader 
35 string 820 is comprised of dm 64 bit string length 822 and the 32 bit dau woni size 82^ Tlie data stream 
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826 then immediately follows, and the infomatioa traditionally camamed in the header hut now contained 
diiectly in die daia stream is repieseoted as the attached dotted line S2S. Another tem used for dm 
flfyy ^M< informMyy!' ts a *shadow channel* as also depicted in Fig. 17. 

Yet anodier dement diat may need to be inciiidfid in the leader string is some sort of complex 
5 check sum bits which can verify diat die whole of the data file » imact and mudtered. This is not iodttded 
in Fig. 17. 

MAPP nM r>f<rn?m(JTED UN TVEI^SAL CODE 5;Y?rTEMS: DYNAMIC CODES 

One intriguing variarion on ihc theme of univcial codes is the possibility of the N-bit 

10 idcndfication word actually coataming instructions which vary the operations of die universal code-system 
itself. Oncofmany examples is immediattly in order: a data transonssiott is begun wiicrein a given block 
of audio dau is fiilly transmiucd, an N-bit ideatificaiion word is read knowing diat die first block of dau 
useduniV€isalcode$i^U5oniofasaof300, say, and diat pan of die N-bit ideoiificatioa word thus found 
is die uistnicriottsdiat the next block ofdaiashoidd be 'analyM^^ using die muversal code sec #4 H ladicr 

15 ihan*145. fa general, diis tcdinology can thus be used as a method for chang^ 

decoding mstnictioos ihemadva. Also in general, diis ability to ndlizc -dynamic codes' should greatly 
increase die sophistication levd of the data vcrificatioa procedures and increase the ccoaomic viability of 
systems which arc prone to less sophisdcaied diwarting by hackei^ and would-bc pirates. The inventor 
docs not bdicvc diat die concept of dynamicaUy changing decoding/decrypting instructiois is novel per 

20 but die eanying of diose mstrucnons on the "shadow channel' of empirical data does appear to be novd to 
die best of the mvencor's understandmg. [Shadow channel was previoialy defined as yet anodua- 
vemacnlar phrase eocapsuiaring die more sieganographic proper etocnts of diis technologyj. 

A variam on die dieme of dynamic codes is die use of universal codes on systems which have a 
priori assigned knowledge of which codes to use when. One way to summarize diis possibility is die idea 

25 of "die daily password. " The password in diis example rcprescoxs knowledge of which set of universal 
codes is cuncndy opcraavc, and these change depending on some set of application-specific diunnstanccs. 
Prcsuinably mimy applications would be coniiflttally updati^ 

before been used, which is often die case wirii die tiaditionai concept of dw daUy password. Part of a 
cnriendytiansmiiied N-bit icfendflcation word c^^ password, for 

30 example. Ttough dmc might be the most ciimmon trigger events for d^ 
could be event based triggers as well. 
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The piacesKOt of jdemificatioa patterns into images is ceitoinly not new. Logos stamped inco 
comers of images. Aibtie paitenu sudi as tnie signatures or dze wailpapenng of ibe copyright eiide-C 
symtni. and [be wateimarlc proper aie all examples of placing pattens tato images in order to signify 
ownership or to tiy to prevent ilticit uses of cfae creative matexiai. 

5 What does appear to be novel is the approach of placing independent 'caiiier'' paizems, which 

themselves are capable of being modnlaxed with certain infotmation« directly into Images sod audio for the 
purposes of transmission and discetnaem of said iofonnaiioa* while effectively being impetceptible and/or 
unintelligible to a pcrcetvsng hi'*^"" Steganographic sohuiaiis cnrxcmly known do tixe inventor all place 
diis infonnaiion "directly" into empirical data (pcKsibly first enoypted, dxen directly), whereas the methods 

10 of this disclosure posit the creation of these (most-often) coextensive carrier signals, the modulation of 
those carrier signals with the information proper, THEN die direct applicatioa to (he empirkai data. 

la extending these concepts one step iunher into the application arena of universal code systems, 
where a sending site transmits empirical data with a certain universai ooduig scheme employed and a 
receiving sice analyzes said empirical data using the universal coding scheme, it would be advantageous to 

15 r^ifg^ a closer look at the engineering consideratxons of such a system designed for the txaoamission of 
images or motion images, as opposed to audio. Said more cicady. dtie same type of analysis of a specific 
implementation such as is contamed hi Fig. 9 and its accompanying discussion on the universal codes in 
anrfin aj^Ucadmis should as well be done on imagery (or two dimensional signals). This section is such an 
analysis and outline of a specific implemenution of universai codes and it attempts to anticipate various 

20 hurdles that sudi a method ^uld clear. 

The unilymg theme of one implementation of a universal coding system for Imagery and modon 
imagery is 'symmetry. * Hie idea driving this couldn't be more simple: a prophylactic against the use of 
image rotation as a means for less sophisticated pirates to bypass any given universal coding system. The 
guiding principle is that die univenai coding ^stcm should easily be read no matter what rotational 

25 orientation the subject imagexy is in. These issues are quite common m the fields of optical character 
recognition and ot^ recogmdon, and dtese Seids should be consulted for further tools and tricks in 
furthering the engineering implementation of this technology. As usual, an immediate example is m order. 

Digital Video And Internet Company XYZ has developed a ddivcty system of its produa which 
relies on a non-symmetric universal codhig which doi^le checScs incoming video to see if the mdivfduai 

30 frames of video itself, die visual dau« contain XY2's owu relatively high security internal signature codes 
»f«ng the methods of diis teehnoiogy. This winrks well and fioe for many delivery siniations, mdutfing 
their Internet tailgate which docs not pass any material unless bodi the hcaxfer infortnaiiDa is veriSed AND 
ibe m-frame unhreisal codes are ficnmd. However, another piece of dieii commercial network peifbrms 
ftiniMigfii^ xouxhie aumlioring on Imemec channels to look for unauthorized cransmissicm of their ^xoprietary 

35 creative properryr. They control the encryption procedures used, thus it is no problem for diem to 
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unenaypi creative propeny, mchiding headers* and perform sKraighdbrwaid diecks. A ptram group that 
warns to izafSc aiaieriaJ on XVZ*s oecworic 2us deternnned how lo modify die secorinr feamres in XYZ's 
header mfoimadon sysrenu and ±ey lia/e fiudiennorc disoovexed due by simpiy cotaiing imagery by 10 or 
20 d^Eces. and cransminmg it over XYZ's network, the network doesn't recogmze the codes and mercfore 
5 does Qot flag illknt uses of dieir materiaL and the receiver of the pirate's rotated material simply unzotates 
it« 

Si^wimanrnig dlis Ust example via logical categories, (he non-symmetric imrrersai codes are quite 
acceptable for die "enafaiemem of amhorized action based on die finding of the codes/ whereas it can be 
somewhat easily by-passed in die case of 'random monitonng (policing) for die presence of codes. * [Bear 

10 in mind diat die non-symmetric univcnal codes may very well caich 90% of Ulidt uses, i.e. 90% of ttc 
illicit users wouldn't botho' even going id the simple by-pass of rotaiiaL] To addness this latter caugoiy, 
die use of quasi-xotatiianaliy symmetric uoiversai codes is called for. 'Quasi'' derives from the age old 
squaring dw ciff le issoe* in diis instance translating into not quite being able to represent a full 
inaementaUy rotationai symmetric 2-D object on a square grid of pixels. FozdieEmore, basic 

15 consideiatioig must be made for tnlp^iagnifirarion changes of die onivetsai codes. It is understood that 
the monitonng process mvst be perfoxmed when die mcnitored visual material is in the 'percepmal' 
domain, i.e. when it has been unencrypted or unscrambled and in the form widi which it is (or would be) 
presented to a human viewer. Would-bc pirates could attempt to use odicr simple visual scrambling and 
unscrambling techniques, and cc3ols could be developed to monitor for dicsc telltale scrambled signals. 

20 Said anoUicr way, would-be pirates would then look to transform visual material out of die percqiDial 
domain, pass by a moniroring point, and dien oransform die material back into die percepraal domain; 
tools odicr dxan die monitoiing for univtrsal codes would need to be used in such scenarios. The 
monitoring discussed tee Utfrefore qiplies to applications wbm momtoiing can be performed in d» 
pcrcepmal domain, such as when it is actttaUy sent to viewing equtpmsot. 

25 The "ling' is die only fall rotaii<»ally symmenic two dmffinsiotial object. The "disk" can be seen 

as a simple finite series of coDce c tr i c and pcrfcctiy abutted rings having widdi aicmg dieir radial axis. 
TTius, die 'ring" cflf^g to be die stating point from which a more lobust universal code standard for 
images is found. Hie ring also will fit nicely into die issue of scale/magnificariott changes, where die 
radhis of aring is asinglc parameter to keep track of and account for. Anodier property of die ring isdiat 

30 even die case where dif&renrial scale changes are made to diffcrem spanal axes in an image, and die ring 
turns into an oval, many of die smoodi and qnasi-symmeoic properties diac any automated moniioring 
system wiU be looking for are generally maintained. Likewise, appreciable geometric distortion of any 
Linage wiUdeariydiston rings but diey can stiU maintain gross syu^ Hopefully, more 

pedestrian mcdiods such as simply 'viewing' imagery wUl be able to detect attempted iflicit pirac/ in diese 

35 t«gards, e^jecially when snch lengdis are taken lo by-pass die universal coding system. 
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jbn^ to Kflota 

Having iHsDOYered tbe ring as the only ideal symmettic pactem upon whose fbiisdaiion a M 
rotattanaUy robust uaivosal coding sysiem can be fmih, we most mm ttus lusic paneni isio something 
{uscdooai, sooetlung vbicb can carry infonnauon, can be read by compuners and ocber insaamencadoa, 
5 can Axivive simple transfoxmaiiotts and ooinipiions, mi can give rise to Rasonably high krutis of secoriicy 
(probably not unteakabie, as the acaian on universal codes explained) in order to keep the ecooonues of 
subversion as a simple tncremeniai cost item. 

One embodiment of the "cing-based* univeml codes b what the inventor rtfa% to as laiot 
patterns'* or simply "knots/ in deference to woven Cdtic knot patterns which were later refined and 
10 exalted in the works of Leonardo Da Vmci (e.g, Mona Lisa, or his knot engravings)* Some rmnon have 
it that these drawings of bxots were indeed steganographic m namre, i.e. conveying messages and 
signatures: all the more i^pn^ate. Figs. IB and 19 eoq)iixe some of die ftmdamental properties of dsese 
knots. 

Vnfo simple examples of kzmt paoenis are depicted by (he supta-iadjai kntns, 850 and die radial 

15 knou852. The names of diese types are based on tbe eemzal synuneiry poim of die spUyed 

wfa^ber the constitueot rings tmeisea this point, are fully outside it^ or in the case of sub-radiat knots the 
central point would be inside a consritusnt ctrule. The examples of 850 and 852 clearly show a 
symmctiitai arrangement of 8 rings or ciicics, "Rings' is die more appropriate ictm. as disauscd above, 
in that diis term explicidy acknowledges the widdi of the rings along die cadial axis of the ring. It is each * 

20 of the individual rings in die knot pattern 850 and 852 which will be the carrier signal for a sin^ 

associated bit plane m our N«bii identification word. Urns, the knot paneroa 850 and 852 each are an S-blt 
carrier of infoimadon. SpcciiicaUy, a«i™tng now dxat die knot pansems 850 and 852 aie hmunous rings 
on a blaci background, then die "addition* of a iominous ring to an mdependent source image could 
represent a *1* and the "sabtraoioa* of a Lumicoos ring ftom an independent source image could rqnncsent 

25 a '0/ The a{q}licatton of (his single encoding scheme could then be repUcated ovct and over as in Fig. 
19 and its mosaic of knot patterns, with die ultimate step of ad(Ung a oled down version of this encoded 
(modulated) biot mosaic dirwtly and cooaensivdy to the originai image, with die resuicam being die 
distri b ^ffMf vhkh has hem encoded via diis universal symmetric coding mcdiod. It remahis to 
communicate to a decodxng system which ring is die least significam bit m our N-bit tdeodfication word 

30 and which is the most significant. One such method is to make a slighdy ascendmg scale of radii values 
(of die mdividnal rings) ixoffl die LSB eo die MSB. Anodier ia to merely madrn die MSB, say. 10% larger 
radius than all die odien and to j^-assign ccsunicrciockwise as the order with which the remaining bita faU 
out. Yet am^er is to put some simple bash mark inside one and only one dxdc. In other words, there 
are a variety of ways with wfaidi the bit order of the rings can be encoded in these knot patterns. 
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A procedure for, firsi, checicmg for tbc mere existence of [hese hiot paacnu and, second, for 
reading of the N-bit identtficaiioii word, is as foflows. A suspect image is first ftmiier tiansfonned via die 
extrtfflety commoa 20 FFT computer procedme. Assuming diat we don't taww the cxaa scaie of die 
kDOC paneios, i.e.« we don't kaow die cadxos of an e i einenfa l ring of tbe knot pattern in die tmits of pixels* 
5 and diat we don't know die exact rotatiooal state of a kzxit pauenu we mcreiy inspect (via basic amomated 
pacem cecogniiion meduds) the resaiting magnitude of the Fourier tnonsfbini of tbe (viggzal boiage for 
teBcale r^ple pactems (coiKentric low ampUtiKle sinusoidal tings on top of die spaisai frequency profiie of 
a souicc image). Tlic periodicity of tbese ring*, aloi% widi die spacing of die rings, wfll iafonn us diat 
die universal knot partems are or arc not likely prBscat, and dicir scaic in pixels. Qassical small signal 

10 detection metHods can be :qpplied to diis problem just as ihey can to die oiha detection merhodoJogies of 
diis disciosuie, Conunon spatial filtering can dien be applied to die fourier iransfonaed suspect image, 
where die spatial Sltcr to be used would pass ail sp«iai frcqucnflcs which are on die crests of die 
concentric circles and block all odier spariai &eqnencies. The resoiting filtered image wcmid be fourier 
tiansfotxned out of die ^dal frequency domaia back uito die . 

15 inspection die inversion or non-inversion of the luminous rings could be d^ected, along widi identification 
of die MSB or LSB ring, and die (in diis case 8) N4rit identification code word could be found, acarfy, a 
pattexn recognition procedure could perform dus decoding step as wdl. 

The preceding discussion and the mcdiod it describes has certain practical disadvantages and 
riiortcomings which will now be discussed and improved upon. The basic mediod was presented in a 

20 simple-minded fashion in order to communicate the basic principles involved. 

Let's enumeraie a few of die practical diiHcultics of die above descrfted universal coding system 
using die knot patterns. For one (1), the ring patterns are somewhat mcfficim m drar "covenog" of die 
fhU image space and in using all of die mformatiottcarr^ 

ring paocms diemselvcs wiU afanost need to be more visible to die eye if diey are applied, say. m a 
25 straightforward additive way m an 8-bit black and white image. Next (3). die -S" rings of Fig. 18, 850 
and 852, is a radier low number, and moreover, diere is a 22 and one half degree rotation which could be 
applied to die figures which die recognition mcdwds would need lo contend widi (360 divided by 8 divided 
by 2). Next (4), stria ovcriiq^ping of ringi would produce highly condensed areas where die added and 
subtracted brightness coidd become cpriic appreciable Next (5), die 2D FFT routine used in die decoding 
30 is notoriously computationally cumbersome as well as some of die pattern recognition mcdiods alluded to. 
FinaUy (6), diough diis heretofore described form of universai coding does not pretend to have ultra-high 
security in die dasskai sense of top Murity conmninicarions systems, it would ncverdieles be 
advantageous to add certain security feanircs which would be mexpcnsivc to impleacm m hardware and 
software systems which at die same time would faicrease die cost of wouki4je pirates anempthig to diwart 
35 die system, ami increase die necessary sophistication level of diose piraies, to die pomt diar a would-be 
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piiate would bave to go so far out of diexr way to thwan che sysiem xhai willMiess would \>q easily 
provea and hopefully subject to stiff crimiiui liability and peaalty (sucb as (he aeasioa and discributioo of 
tools wbich scrip creative piopetiy of these knot panem codes). 

Ail of these items Can be addressed and should continue to be refined upon in any eagioeering 
5 iznpleme&tatton of the principies of tbo tedmoiogy. This disdosuie addresses diese items widi reference to 
die following endnxiiinenis. 

Riynnrng with ItcxD uumbcT 3. that diere are only 8 lings represented in Fig. 18 is sin^jiy 
remedied by aicrearing die number of rings. The number of rings that any given appiication will utilize is 
clearly a function of the applicatioo. The txade-o& inchide but axe not limited to: on the side which 

10 argues to limit the number of rings utilized^ there will ultimateiy be mcite signal energy per ring (per 
vistbllicy) if there are less tings; the rings will be less crowded so that diere discernment via automated 
lecognition mcdsods wilt be facilitaied; and in genexai smce they are less ciowdedi the full knot patiem can 
be ^tained using 1 smailcT overall pixel extern* e.g. a 30 pixel diameter icgion of image rather than a 
100 pixel diameter region. The arguments u> Increase the number of tings include: the desire to transmit 

IS moie information, sueh as ascii infoimation, serial numbers, access codes, allowed use codes and index 
nnmben, history inlbrmatiotu etc.; another key advantage of having mote rings is liiat the rotation of die 
knot pattern back into itsdf is redticed, dierdiy allowing the recognitian Trrrhod.s to deal widx a smaller 
range of cotatioa angles (e.g., 64 rings will have a maximum roiatiooai displacement of just under 3 
degrees, i.e. maximally dissimilar to its original pattern, where a rotadon of about 5 and one half degrees 

20 brings the knot panem back into its initial alignment; the need to diadnguish die MSB/LSB and the bit 
plane order is better seen in diis example as well). It is antic^ated dial most ptacdcal appltcattons will 
choose between 16 and 128 rings, conesponding to N=16 to Ns 128 for liie choice of die number of bits 
in die N-bit identification code word. The range of this choice would somewhat coixeiate to die overall 
radius, m pixels, allotted to an elemental knot pattern such as 850 or 852. 

2S Addressing the practical difficulty item number 4, dtat of die condensation of rings paneins at 

some paints in die m^gip and l2ck of ring pattexits iu oihen (which is very simiiar, but still distint:: fttnn. 
item 1, me ineffident covering), the following improvcmem can be a^^lied. Fig. 18 sbows an example of 
a key feature of a "knot" (as opposed to a pattern of rings) m that where panems would supposedly 
inteisca* a virtual third dimension is posited whereby one strand of the knot takes piccedeoce over another 

30 strand m some predefined way; see iiem 854. In the ceims of imagery, die brigbtnsss or dimness of a 
given intersectioa point m the knot patterns would be "assigned* to one and oxdy one strand, even in areas 
wfaete more tban two strands overlap. The idea here is dien extended, 864» to how rales about dns 
iwj ftgnmw^t shonld bc carried om in some totatzonally symmetric manner. For example, a nile would be 
that, travelling dockwise, an mcoming strand to a loop would be "behind" an outgoing strand. Clcariy 

35 tbez6 are a mil""^^** of variations which could be applied to diese rules, tnany which would critically 
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depoid on die geomeny of the knot paoexns chosen. Other issues involved will probably be that the 6nue 
width, ami inoiGover the brightoess profile of the width along the oominl axis to the direction of a strand, 
will all play a lole in ibe rules of bri^cness assignoient to any given pixel uoderiyisg the knot pattens. 
A major improvement to die nominai knot patum system previously described dizectly addresses 
5 practical difficoitks (!}« the inefBcient covering, (2) the unwanted visibility of the cinp, and (6) the need 
&r higher levels of secnnty. This i mpro v em e nt also mdirectiy addiess ton <4) the overiapping issue, 
which has been i^i r'^gg.i^ in the Last pazagnph. This major ixnpfovenm is the following: jnsi prior to the 
seep whext die mosaic of die encoded knot pattens is added to an origind mag^ to produce a discribtitable 
image, to mosaic of encoded knot patterns, 866, is spaiiaUy filtered (using common 2D FFT tedmiqaes) 

10 by a standardized and (generally smoothly) random phase-only ^atial filter. It is very important to note 
that this phase-only filter is Itsdf fully rotationally symmetric withm the spatial frequency domain, i.e. its 
fikcring effects arc fully totaiioaaily symmntik. The effect of this phase-only Sltcr on an nidividual 
luminous ring is to transform it into a smoothly varying panera of conccaxric rings, not tzxally dissimiiar to 
the panein oa water several instances att/ex a pebble is dropped in, only that the wave patterns are 

15 sonxewrtiai random ui die case of this phase-only filter raifacr ±an the unifoim periodicity of a pebble wave 
pattern. Fig, 20 attempts to give a nragh Ci.c. non-greyscak) dcpictian of these phase^y filtered ring 
patKOS. Hie top figure of Fig. 20 is a cross section of a typical brighmess contour/profile 874 of otc of 
diese phase-only filteted ring patterns. Referenced in the figure is the nominal location of the prc-filtered 
outer ring center. 870. The center of an individual ring, 872, is referenced as die point around which the 

20 brighmess profile is rotated in order to fiilly describe the two dimensional brightness distributioa of one of 
these filtered patrems. Yet another rough attempt to mtnmnnTraTr the duracterisncs of die filtered ring is 
depicted as 876. a crude greyscaie image of the filtered ring. This phasc-onJy filtered ring, 876 will can 
be refened to as a random ripple pattern. 

Not depicted ui Hg. 20 is the composite effects of phase-only filicring on die knot paitons of Fig. 

25 18,oronthemofiaicofbiotpatiansa66inFig, 19. Each of die individual ris^ m die knot patterns 850 
or852willgivcrisctoa2D btighmess paitem of the type ff76. and together they fonn a ratho" 
compUcainl brightness pattern. Realizing diat the encoding of die rings is done by matong i^ 
or -anti-luminons* (0). the tesaliing phase-only filtered knot partems begin to take on subtle characteristics 
which no longer make dirta sense lo the hmnan eye, but ^ch are stUl readEy disccmablc to a conqnucr 

30 especially after the phasc-oniy filtcmg is inverse filtered reprodudog the original rings patterns. 

Rauming now to Fig, 1 9, we can imagine diai an Wiit identification word has been encoded on 
the knot patterns and die knot patterns phase^)nlyfiltEiwL The resulting brighmess distribution would be a 
rich tapestry of overiapping wave pattern fl*idi would have a certain beamy, but would not be tcadUy 
inteUigibletodweycrtjraia. [An exception to diis might dfiw 6offl the fore of die South Pacffi^ 

35 ccmmmnides* where it is said diai sea oaveflen have learned Uic subtle art of readmg small and multiply 
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compiex oceaa wave patterns, generated by ^Hfraaed and reflected ocean mvei off of ioterveoisg isLmds, 
as a primaiy siavigaiionai tooL} For wane of a better tenn, the resulcing mosaic of filtered knot pattenu 
(derived from 866) can be called the encoded knot tapestry or jost the knot tape^. Soioe basic 
propeities of diis knot tapestry are that it renrins the .base notational symmeny of its gei^nuor mosaic, it is 
5 generally unistelligible to the eye/tiain, thus raising it a notch on the sophistication level of revene 
engineenng, it is more efiident at using the available tafbnnadon content of a srid of [nxeis (more on diis 
in the next secckm)* and if the basic knot concepts 854 and 864 are utilized, it will not give rise lo local 
"hoc spots' where die signal levd becomes undnly condensed and hence objecdonably visible to a viewer. 
The basic decodmg process previously described would now need die additional step of hxverse 

10 filtenng die pbase-ooiy filter used in the encoding process. This inverse filtering b quite well known in 
die irna^ processing industry. Provided th^ die scale of die knot patterns are known a priori, the invene 
filtering is straightforward. If on the other hand the scale of die knot patterns is not known, then an 
additional step of discovering this scale is in order. One such method of discovering die scale of the knot 
pattens is to Iteiatively apply die invene phase-only filter to vaiiously scaled version of an image being 

15 srarching for which scale-version begins to exhibit notkeable knot paneming. A canmian search 

aigodlhm stich as die smo^ex medtod could be used in order to accurately discover the scale of the 

The field of object recognition should aiso be consulted^ under die general topic of 
uoknown-scaic object detection. 

An additional point about die efficiency with which die knot tapestry covers die image pixel grid 

20 is in order. Most applications of the knot tapestry method of universal image coding will posit die 
applicadon of die ftdly encoded tapestry (i.e. die tapestry which has die N-bit identification word 
embedded) at a relative low brightness level into die source image. In teal tetsas» the bnghmess scale of 
the f n r ff^H tapcsixy will vary &onu for example, -5 grey scale values to 5 grey scale values in a typical 
256 grey m\ f image, where die preponderance of values will be widiin -2 and 2. Una brings up the 

25 purely practical matter dia the knot tapestry wiU be subject to appreciabk bit irnn^ Pat as an 

e3cam|rie» imagise a ocmstmcted knot tapestry nicdy uniizix^ a full 256 grey level image, then scaling dils 
down by a faoor of 20 in brightness indnding die bit truncation step, then rcscaiing diis truncated version 
back up in bri^imess by the same Eaoor of 20> dien inverse phase-otdy filtering die resuliam. The 
resulting knot partem mosaic wili be a noticeably degraded venion of die original knot pattem mosaic. 

30 Tlie point of bringing all of dtis up is die R)12owing: it win be a simpiy defined* hut indeed challenging, 
engineering task to scica die various free parameters of design in die implesiemadoii of die kimt tapestry 
medmd, the end goal being lo pass a amount of Information about die ^f-bit idendfication word 

widimsomepie-defined visibility tolerance of die knot tapesiiy. The free parameters bdude but would 
not be fully limited to; die radius of die elfmemai ring in pi;(els, N or die cumber of rings, die distance in 

35 pisb fram du center of a iooot paaem to die center of an elemeniai ting, die packing criteria wd 
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distances of one knot pattcm widi the orhcn, cfic rules for snand weaving, and liie foims and types of 
piise-only Qltm to be used on tbe kziot xoosaics. It would be desirable co liaed such pacameien inm a 
compmer opdmizoioii routme whicb could assist in their sdection. Even tfaii would begin sniely as more 
of an an thsa a science due to the many oon-luiear fiee parameters involved, 
5 A side note on the use of pfaase-oniy filtering is tbat it can assist in dK detecdoa of die rii^ 

paneras. It does so in diat tbe inverse filtering of die decoding process tends to ^biur" die underiying 
source ^^^ p upon whidi die biot upestiy Is added, while ac die same dme 'bringing into focos" the ting 
panenis. Without dw Wuning of die source image, die emerging ring patterns would have a harder time 
•compcting" widt die sharp fcamrcs of typical images. The rtrcoding proccdnrc should also utilize the 

10 gradient ducshoiding mcdiod described in anodicr sectioa. Briefly, diis is the racdiod where if it is known 
diat a source signal is much larger in brightness dian our signanirc signals* dien an unage being decoded 
can have higher gradient areas duesholded in die service of increasing die signal levd of die signamte 
signals reladve to the saaax signal. 

As fbr die odier pracdcal difficulty meoiioned earlier, item <5) which deaU widi die rdative 

IS computational overhead ofdie 2D FFT routine and of typical pattern re^^ 

here posited but not filled is to fmd a ainqiler way of quickly iccognizLog and decoding die polarity of dK 
ring brightnesses than diat of using die 2D FFT. Barring diis, it can be seen diat if die pixel extent of an 
individual knot pattern (850 or 852) Is, for exampie. 50 pixels ia diameter, dian a stmpie 64 by 64 pixel 
2D FFT on some secdon of an "unage be more dian sufficient to discern die N-bii identification word 

20 as previously described. Tlie idea would be to use die smallest image region necessary, as opposed to 
being required to utilize an enure image* to discern die N-bit identification word. 

Anodier note is diat diose pracritioneis in die science of inagc processing wiH recognize diat 
instead of beginning die discussion on dje knot tapestry widi die utilizarion of rings, we could have instead 
jumped right to die use of 2D brighmess distribution patterns 876. QUA bases ft^^ Tbeuseofdic 

25 "ring" tcmnndogy as die bmdine technology is partly (fidactic. as is appropriate for paicnt disclosures in 
any cvem. What is more in^ortant, perhaps, is diat die use of true 'rings" in die decoding process, 
posi-invasc filtering, is probably die amplest form to input into typical pattern iccognition routines. 



30 



Mmirit MPWflTif Decoders 

Those skilled in die rignal processing art wiU recognize dial compuieis employing neural network 
aniitecnirta are well suited to die pattern zecognidon and detection^f-small-signal-inHioise issues posed 
by die ptesem technology. Whik a complete discourse on dieae topics is beyond die scope of diis 
spedficanon, die interested reader is referred to, eg., Chcricassky, V., -fmni ,Sratisnc?? to Neuyal 
^^ri«- TTieoiv i^ni Recomidott AooHcations/ Springer-Veriag. 1994; Masters, T., -SiSSaLfe 
35 ^r- r ^' " ^ f ^ r^' ^'^'^"^ ^ Snnrrebook / WUev, 1994; Guyon, 1, ^ Advances in Pattern 
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RecQgnition Svsteim Using Kairal Networks / World Scientific Publishers, 1994; Nigrin. A., " Neural 
Networks for Patteni Rer pCTinon/ MIT Press, 1993; Gchoki, A., * Neural Networks for Opiimization & 
Signal Processiag ." Wiley, 1993; and Ctei, Cm " Neural Networta for Pattern Recognition Ai Their 
Apolicatioiis/ Wodd Sdcmigc Poblisbers, 199L 

5 

3D UNnVERSAL CODES n : SIMPLE SCAN UNE IMPLEMENTATION OF THF ONE 
DIMENSIONAL CASE 

The above sectioa on rings, knots and tapestries cenainly has its beauty, boi sons of the steps 
involved may have enough complexity chat practical iroplementatioos siay be too costly far csnaJn 

10 applicoiioss. A poor cousin the concept of rings and wdkksigned symffletzy is to simply utilize the basic 
concepts prcscmcd in conneaion with Fig. 9 aod the audio signal, and apply thetn to two dimensional 
signals such as images, but to do so in a manner where, for example, each scan line in an image has a 
random stamng point on, for example, a 1000 pixel long universal noise signal. It would then be 
uiozmbeot upon rccogninon software and haidwaze to intetrogate imagery across the foil range of 

13 rocadonal states and scale facton to "Gnd" the existence of these univeisal codes. 

THE UNTVERSAL COMMERCIAL COPYRIGHT OJCQ IMAGE. AUDIO. AND VIDEO FILE 
FORMATS 

It Is as wdl known as it is regretted ihat there exist a plethora of file fonnat standards (and 
20 not-so-siandards) for digital images, digital audio, and digital video. These standards have generally been 
formed within specific mdustncs and applications, and as the usage and exchange of creative digital 
material pEcliferaied, the various file formats slugged it out in cross-disciplinaiy arenas, where today we 
find a de facto histogram of devotees and users of the various favorite formau. The JPEG, MPEG 
standards for foimamg and compression are only siighi excepdoos n would seexti, where some concerted 
25 cioss-mdustry coUaboraiioa came into play. 

The czy fbr a simple universal standard file format for aucUo/yisufll dau is as old as the hiHs. The 
cry for the protection of such oaterial is older still. With all due respect tb the inoate dif&cnliies anendam 
upon the creation of i onivenal format, and widi all due respea to die pretentiousness of outlining such a 
plan within a patent disclosure, the inventor does b^eve diai these methods can serve perhaps as well as 
30 anything for being the foundation up<m which an accepted world-wide "universal cocnmeicial copyright' 
format is built. Practitiotiers know that such animals are not built by proclamadon, but through die 
efficiem meeting of broad needs, tenacity, aod luck. More germane to die purposes of this disdosuie is 
the fact that the appUcatxon of this technology would benefit if it could become a central piece within an 
industry standard file fdnnat. The use of unxvenai codes in particular could be specified widiin such a 
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standard. Tlie MJest expression of the commercial usage of ihis ledmology comes from the icnowlcdgc 
that Ehe invisible signing is takmg place and the conlidCTce that instills in oopyii^ holden. 

The following is a list of reasons dm the prxxsciples of diis cechnclogy could serve as the catalyst 
for such a standard: (l) Few if any tcconicaJ deveiopnienis have so isoiaied and so pointedly addressed die 
5 issue of bioad-brash protection of empiiical data and audio/Visual naietial; (2) All pievions file formats 
have treated die infotniadoa aboot die daia, and die data iisdf. as iwo separate and pfaiysically distinct 
emicies, whereas die medtods of dds cscbnology can combine die two inio one physical entity; (3) The 
mffis scale application of die principles of dus technology will require snbstaniiai standardiaaiion wotk fai 
die fint place, iTifl !T^^T? g imegiadoa with die years-to-come im^Hfovemfims in compression technologies, so 

10 die standards kfosinicmrB will exist by dcfedt; (4) die growdi of multimedia has creared a generic class 
of data caUed 'content/ which mcludes text, images, sound, ami gKq^liics, arguing for higher and hi^ 
levels of "coBtcni standards'; and (5) marrying copyright protection tedmoiogy and security feainrcs 
direcdy imo a file formal standard is long overdue, 

Elements of a universal standard would cenainiy mchule die mirroring d^cas of die header 

15 verification mediods, where header information is verified by sxgnanire Also, a 

miivetsal standard would oudine how hybrid uses of fully private codes and pubOc codes would 
commingle. Thus, if die public codes were "snipped" by sophisdcated pirates, die private codes would 
remain mtact. a miiveisal standard wodd specify how invisible signatures wouM 
and audio evolve, TTjus. wheii a given image is created based on several source ^ 

M would spcdfjr how and when die old signatures would be removed and replaced by new signamres. and if 
die header would keep track of dicse evolutions and if die signatures dwmscivcs would keep some kind of 
record. 

PTYF15VS, BUMPS 

25 Most of the disdosarefbai^ Oil pixdi being 4c basic cam 

•n» jecdon diaausing tense of a sli«le •aasier coda agjial' wrai so &ras to easajMlly 'aasiga" ea* 

ad CTCty pixel to a oaipe bit plane oi ihe N-bit idradficarion word. 

For osany appUcattons. with one exemplar being tha of ink based priniiBg ai 300 dots per inch 

resolution, what WM OKC a pixel in a ptisane digital image file bcames effotively a blob (e.g. of 
30 dithered inlt on a piece of paper). Often the isolated inHonnaiion cnnying capacity of ibe original pixel 

becomes comptomised by neighboring pixels spilling over kta the geometricaUy defined space of the 

original pixel. m« practicaJ in die m wiU lecognize ihis as simple spatial fUteri^ 

binning. 

b juch citeainstaBces it may be more advantageous to assign a cenain highly locai gronp of pixels 
33 tt,a«iiijBebitplaactoti»N4>iii<Ieatificationword,iaiherth^ 
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simply to prc-concoitratc mote of the signamrc signal energy inio the lower ftequcndes, realizing tbat 
azost praciicai impleinentauoiis qaicJdy strip or mitigaie higber frequencies. 

A simplc-oinded approach wouid be to auign a 2 by 2 black of pixels all to be modulaiEd with 
the same almnate signamxe grey ralue^ rather than moJuiating a single assigned pixel. A more fancy 
5 approach is dqsicted m Figs. 21A and 21B» where an arr^ of pixel groups is depicted. This is a specific 
sample of a large class of configurations. Hie idea is that now a certain small region of pixels is 
assocxaed widi a girai onique bit plane in the N*bii idendlScation wctfd, and that this grouping actually 
shales pixels between bic planes (though it doesn't necessary have to share pixels, as in the case of a 2x2 
block of pixds above). 

10 Depicted in Figs. 2IA and 21B is a 3x3 array of pixels with an example normalized weighting 

(normalized -> die weights 2Kid up to 1). The methods of this technology now operitfe on diis dementary 
"bump, ' as a onit, rather than on a single pixel. It can be seen that in diis example there is a fourfiold 
decrease in the number of master code values that need to be scored, due to [he spreading out of die 
signature signal. Aj^iicatinns of this "bump approach" to piachzg m invisftle signatures inchide any 

15 application which will experience a priori known high amounts of bhirring, where jmiper identification is 
still desired even after diis heavy bhining. 

ft/ ^^PP nN STEGANOGRAPHTC USES OF THIS TECHNOLOGY 

AS mentioned in the <nmai sections of the disclosure, stegaxxography as an an and as a science is a 

20 generic prior an lo this technology. Putting the shoe on die other foot now, and as already doubtless 
apparent to the reader who has vennxred thus far, the methods of this technology con be used as a novel 
method for performing steganography. (Indeed, all of the discussion ihcs fiur may be regarded as 
exploring various forms and impleoientaiiotss of steganography.) 

In die present section, we shall conaidsr steganography as die need to pass a message firam point 

25 A to poim B, where due message is essenttaUy hidden widiin genexaUy indep^^ As 
anytme in die mdustxy of tdeoonmsmicadons con attest to. die rangp of purposes for passing messages is 
qnile broad. Presumably diere would be some extra need* beyond pure hobby, to place messages mto 
empirical data and cmpiiicai signals, rather than sending those messages via any number of conventional 
and straightforward" dianods. Past litcramrc and product propaganda within steganography posits diat such 

30 an exua rr^, among oUiers, might be the desire lo hide the faa that a message is even being sent. 
Anodier pos^bte need is that a conventional commnrfnttions channel is not available diiecdy or is cost 
prohibitive, assuming, dsat is, that a sender of mcssagm can "transmit* dieir encoded entpixical data 
somehow. This disclosure includes by reference all picvious discussicms on the myriad nses to which 
steganography mi^ apply, while adding die following uses which the inventor has not iseviously seen 

35 described. 



SUBSTITUTE SHEET (RULE 26) 



. FCnUS97/083Sl . 

WO 97/45736 



63.- 



10 



15 



The first sisdx use is very simple* [t is the need to cany messages about ihe empirical dau wichia 
which die xnessage is eanied. The lide/oke \s that now die media is troiy the message, though it would 
be next lo iopossibte chat some pievioos sttgaoDgrapher hasn't already exploiied dxis joke. Some of die 
discussion on placing information about die empiricai dau directly iiuide that enprical data was already 
covered in die section on iqiiacing die header and die concept of die '^bodiff This section expands upon 
diat section somewhat. 

T]ie advantages of placmg a message about empirical data dtiectly in diat data is chat now only 
one class of data object is present rather than die previous two da«es. In any two class system, diere is 
the risk of die two classes becoming disassociated, or one dass conupicd widsout die odier knowing about 
il. A concrete example here is wtar the invottor refers to as "device inffiyradeat insmicnons. * 

There exist ziliioos of yn^irhir^^ data formats and data file formats. This plediora of formats has 
been notorious in its power to impede progress toward univetsal dau exchange and having one machine do 
the fftnff rhing that anodier machine can do. The instruaions diai an originaior might put into a second 
class of dau (say die header) may noi at all be compaiible widi a machme which is imcndcd to lecogmze 
diese instructions. If format conversions have taken place, it is also poss&le dm critical Instructions have 
been stripped along die way. or garbled. TTic improvements diadoscd here can be uad as a way to 'seal 
in' certain insmicrions direcdy into empirical data in such a way dwt all daa is needed by a reading 
machine to recognize instructions and messi^es is to perfonn a standardized "rccognititm aigoridm' cm die 
empirical data (providing of course diat die machine can at the very least •read* chc empirical data 
propcriy). AU marfiincs could implcmeni diis algoridan any old way they choose, using any compilers or 
imemal ^^ara formats that they wane 

Emplementation of dtis device mdependent mstnicrion mediod would generally not be concerned 
over die issue of piracy or illicit removal of the scaled to messages. Presumably, dm embedded messages 
and instructions would be & eentrM valuable compwicot in the basic value and fancdocing of die material. 
25 Anodxer example of a kind of steganographic use of die technology is the embedding of universal 

use codes for die benefit of a user community. The "message- being passed could he singly a registered 
serial number identifying ownership to usen who wish to legiumaidy use and pay for die empirical 
infonnatiott. The serial number could mdex imo a vast registry of cxcaive pw^ 
or names of die o;mcis, pricing mformation.biIli^ The "message" could also 

be die ctearuM of toe and public use for some given material. Similar ownership ideniificaiifin and use 
indexing can be achieved in two class dau strtmnire mcdmds su^ 

class system of ±ia technology may offer certain advamagcs over die two class system in dm die single 
dass system does not care abom file format conversion, header compatibiUdcs, internal data fornoi issues, 
header/body archiving issues, and media uassfonnations. 



20 
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Piillv RTagt .^anograohv 

Prior art steganograpiilc metliods carrcmiy Icnowu lo the invencor generally involve fliHy 
dcienninistic or •exact* prescriptions for passing a message. Another way to isy this is that it is a basic 
assumption to for a given message to be passed correctly in its entirety, the receiver of the infetinaiion 

5 needs to receive the ex^a digital data file sent by ihc sender, tolerating no bit erron or Moss' of data. By 
dcfimiitsu "lossy' compression aod decompfession on einpitical stgoals defeat such steganogrqihic 
methods. OPrior an, snch as die previously noted Kooatsawoi^ 

Hie pdjxdpies of diis ttchodogy can also be utilized as an exact fium of ssganography proper. 
It is suggested that ssch exact fonos of sieganograpliy, whedKr diose of prior an or diosc of this 

10 technology, be comlnned with die relatively recent an of the "digital sigaawre" and/or the DSS (digital 
signanirc standard) in ajch a way diat a receiver of a given empirical daa file can fini veri^ thai not one 
single bit of inforamion has been alracd in the received file, aod dms verify that die contained exact 
st^anographic message has not been altered. 

Hie simplest way to nse die prindplcs of dm technology in an exaa steganographic system Is to 

15 utilize die previously di^nisscd "desigQed' master noise scheme wherein die master snowy code Is net 
allowed to ooniainiBios. flodi a scirier and a receiver of infcrmatioft would need access to BOTH d» 
aaner snowy nrf" fig"«' ^ origijai unencodcd oripnal signal. The receiver of die encoded signal 
merely lubtracts dw original signal giving die difference signal and die tcchmques of simple polarity 
chcckdag between die diffoeace signal and die master snowy code signal, data sample to data sample, 

20 produdng a die passed message a single bit at a time. Presumably data samples widx values near die 
•rails" of die grrfy value rai^ would be skipped (such as die vahxes 0,1,254 and 255 in 8-bit depdi 
empirical data). 

^atisn cn^ S(g^ographv 

25 Tic need for die receiver of a steganographic embedded data file to have access to dw original 

signal can be renwed by nnnhig to what die inventor refers to as "stadsdcdfl^^ ladiis 
approach, die mediods of dus technology are apjdied as simple a priori rules goveming die reading of an 
empiric^ data set searching for an embedded message. This medicd also could make good use of it 
ooobtnation widi jiior an mcdiods of verifying die integrity of a data file, snch as widi the DSS. (See, 

30 e.g., Walton. 'Image Amhenrication for a Slippery Ncrw Age/ Dr. Dobb's Journal, April* 1995, p. 18 fbr 
mediods of verifying die sample-t^-sample, bii-by-bit, haegricy of a ffi giial image.) 

Stanstical steganography posits dias a sender and receiver bodi have access to die same master 
snowy code signal. This signal can be cnrirely random and securely transrnitied to bodi pa^ 
generated by a shared and securely transmitted tower order key which generates a larger quasi-nrndom 

35 master snowy code signal. It is a priori defined diat 16 Wl dnmlsof a message wiB be passed widi^ 
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coiuxguous 1024 sample blocks of eopixical daca, and chai the receiver will use dot piodnct decoding 
methods as outtined is thbdisclOBUit. The 3CQder of the infannaticm pr&cliedts that the dot pc^ 
appioadi indeed pcoduces liie xctirate 16 bit vahies (that is, the sender pre-cbecks chat the cross^taik 
between die canier image and die mesuge signal is doi sach diat the dm produce operation will produce an 

5 unwantcdioversionof anyof die 16 bits). Sonie fixed miniber of 1024 sample blodcs axe transmiued aixi 
the same mmiber dmes 16 bits of message is therefore transmined. DSS techniques can be used to verify 
the imegri^ of a message when the craosaitted data is known to only exist m digital fonn, whereas 
internal chedcsnm and cirDT cottecting codes can be transnriited in simatioas where the dau may be 
snbjea to change and transfoniiati0n in its transmission. In diis latter case, h is best to have longer blocks 

10 of san^Jles for atiy given message content size (such as lOK samples for a 16 bit message chunk, pureiy as 
an example). 

Comimiing for a nmmcac on the topic of error cotrecring steganogiapiiy, it will be recognized that 
many decoding ificfeniques disclosed henin operate on die principle of distinguishing pixels (or bumps) 
which have beeA augmieQted by die encoded data ftom those chat have been dimini^ by die encoded 
15 data. The disiinguishxng of tfwse poaiiivB and negarivc cases becomes inoeasingly difficult as the delta 
values (e.g. die diffemice between an encoded pixel and die conesponding original pixel) approach zero. 

An analogous sittwiion arises in certain modem transmission tcdmiqacs, wiicrcin an ambiguous 
middle ground separates the two desired signal staas (e,g. -h/- I). Errors deriving &om incorrect 
intcrprttatioo of this middle ground are sometimes termed "soft errors/ Principles &om modem 
20 technology, and odier tcchnologiea where such pioblcms arise, can likewise be applied to mitigadon of 
such errors in tb& present contexL 

One approach is to weight die 'coofideace' of each delta detenninarion. If ibe pixel (bump) 
eleariy rflpm one state or the odier (e.g. +/- 1), its "confidence* is said to be high, and it is given a 
proportiomttely greater weighting. Conversely, if die pixel (bump) is relatively andngucus in its 
25 interpretation, in coofidcace is commcnsurarely lower, and it is given a ptoporuonaiely lesser weighting. 
By weighting the data from each pixel (bunp) in accordance widi its confidence value, the effects of soft 
crron can be gready reduced. 

The ftjiegoing procedure, dworcticaUy sin^>ie, relies on weightings which are best 
determined empirically. Aaamfingly. such an ^loiwA is not necessarily 

An alternative approach is to assign confidence values not to iateipittaaons of individual pixds, 
bmradwr to dcternnnation of bit values -eidier from an image exi^ ^ 
snch an anangemeni. each decoded message bit is given a confidence value depending on die ambiguity (or 
not) of the unage statistics by which hs value was detenrnned. 
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Such coDfidesce weighting can also be used as a belpgil adjuKt with other error 
detectmg/contctmg schemes. For example, in known cmir correcdng polynonuals. the above-detailed 
weighting parameters can he used to fiutber hone potynondai-based discernment of an error's locattoo, 

5 THE "NQBE- IN VECTOR GRAPHICS AND VERY>LQW^RDER IN DEXED GRAPHICS 

The methods of dns disdosur generaUy posit die existeaoe of *empincai signals/ nrhich is 
anodier way of saying signals which have noise cooiained whUa diem afanosi by de&udon. There axe two 
classes of 2 dimensional gr^hics which are not generally considered id have noise nhciem In tbem: vcdor 
graphics and certain uidexed bit-mapped graphics. Vector gr^Uiics and vecKv graphic files are generally 

10 ffles which contain exact butxucdofls for how a conpiter or prio^ A 
change of even one bit value in such a 51e might change a circle to a square, as a very cnide example. In 
other words, dbeie is gesexally no "ichenmt noise* to cxpiaii within chese files. Indexed bit-mapped 
grapliics Kfcrz to im^es which are composed of generally a small number of colors or grey values^ sodi 
as 16 in the eady CGA di^lays on PC computers. Sudi 'veiy-Jow-order' bit-mapped images osnaily 

15 display graphics and canoons* rather than being oscd m dm attempted di^lay of a digital Image takai with 
a caniEca of die natozad worid« These types of vczy-low-ortlcr bit-raajqwd graphics also are generally not 
considered xo contain *noise" in die classic sense of that term. The exception is where indexed graphic 
files do indeed attempt co dqiict samnd hnagety, such as with the GIF (graphic interchange fonnas of 
Cnnpuserve)« where the concept of *naxse'* is still quite valid and the principlea of this technology still 

20 qnite valid. These latter forms often use didiering (similar to pointillist paintmgs and color newspaper 
prim) to achieve near lifelike imagery. 

This section concems this class of 2 dinsnsional graphics which traditionaily do not contain 
"noise." This secnoa T^»cf< a brief look at how the principles of this tedmology can still be applied in 
some fashion to such creative material. 

23 Hie easiest way tc a^ly die principles of diis techrmlogy to diese 'noiseless' graphics is to 

conven them into a form which is amenable to the plication of the princ^des of this tedmology. Matqr 
tenns have been oscd in the hidnstiy for this conveision, "^"H"s *rippmg' a vector gnq>hic (raster image 
processing) such diat a vector graphic file is convened co a greyscale pixel-based raster image. Programs 
such as Photoshop by Adobe have such mtemal tools to conven vector graphic files into RGB or greyscale 

30 digi^ images. Once diese files are in such a form^ the principles of diis technology can be applied m a 
acraighcfbrward manner. Likewise, very-low-indexed bitmaps can be convened to an RGB digital image or 
an equivalent. In the RGB domain, the signatures can be applied to tbt three ocxlor rhannris m appropriate 
laxiaSt or the RGB Image can he simply canvened into a greyscale/chrama fioimat sudi as "Lab" in 
Adobe's Photoshop software, and the signatures can be applied to the 'Ughmess charme!" therein. Since 

35 most of die distribution media, snch as videotapes, CD-ROMs, MPEG video, digital images, and pnnt are 
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all in fonns which arc ameaabJc to the application of the principles of this technology, this conversion 
ftoni vector graphic form and very-low-onlcr graphic fonn ia oncn done in any event 

Another way to apply the principles of this technology to vector graphics and very-bw-oider 
binnappcd graphics is to recognize that, hidccd, there are certain properties to toe hiherent graphic 

5 formars which • to the eye - appear as ociac. The primary exanapfc is the bordco and conwors between 
where a given line or figtne is drawn or nm drawn, or exactly where a bit-map changes 
blue. In most cases, a himwn viewer of such gr^Jhxcs will be 1^ •modulate 
signamre signab" via the efctailed and mediodicai changing of the precise contours of a graphic object. 
Nevertheless, such encoding of the signanucs is indeed possible. The distinciioQ between this approach 

10 and diat disclosed in the bulk of this disclosure is ihar now the signamies tsmi uiamaidy derive from what 
already exists in a given graphic, rather than being purely and separately created and added into a signal. 
TTiis disclosure points out die possibilities here nonetheless. The basic idea is to asodujate a comotti, a 
tonch right or a toudi left, a touch up or a touch down, m such a way as w cofflanmcaie an N-bit 
identification word. Tte locations of d» changes concoars would be contained in a 

15 noise image, though now die noise would be a recod of random spatial shifts ooe diiecdon or another, 
perpendicular 10 a given contour. Bit values of die N-bit identificadon woid would be encoded, and read, 
using die same polarity dieddnsrnediodbetweci^ the applied change and die change recorded in die master 
noise image, 

20 yt Avrrr r»gPiT nt-RTT CAR n .svynrMS BASED ON THE PRINCTPLB OF THE 

Growih in the use of plastic credit cards, and mote receaiJy debit cards and ATM casli cards, 
needs little introduciion. Nor docs ihae need to be audi discussioa here abow tlie long 1^ 
aadiaidt«sejofttaefinandalin$Baincna. lie deretepnew of ihe eedii card hologram, and iu 
25 nhae(ioeiiIforge«yde«lo{mM£. nicely serves as a h^ 

secutityineaswaandftaiidtUentcoviniernieasnres. Tliis seaion wiD coiuxm itself with how the principles 
of dds lectaifltogy can be tcaliad ia m alternative, faigbly ftaud-proof yet cost effective plastic caid-based 
fj^Mff-iai oecwoik. 

A basic Ust of desired feamres for an ubiquitous plastic economy might be as fWlows: 1) A given 
30 plastic Baaaeial card is completely impossible to forge; 2) An attempted fbrged card (a look-alike') 
camio. even flmaion within a cnaaattioB sett^^^ 

would not fa any way be useM or leniseable: 4) la the cvem of physical thefl of an aem^ 
d«c axe stiU foimjdabte obstacles to atof using that card: and 3) ^ 

l,,,pto«ntatiottoftheliiiand^ 
33 card networks. i.e.. the fblly loaded cost per transaction i, ecpial to orless than the current norm, allowing 
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fct Wg hy proa margins to ihe iinjlcmeniins of the networks. Apan from item 5. wbidi would require a 
<lctaiie<l analysis of cte engineering and sodai issues involved with an all out implementation stna^, tlte 
CbUowing use of the principles of this tedmology vuv wdl addeve die above list, even item 5. 

Figs. 22 tbPJUgh 26. along widi the ensuing written material, cuHectively oodine wliai is refcned 
to in Ftg. 26 as "He Negiigible-Frand Cash Card System.' The reaum (bat the ftaud-prevemiott aspects 
of die system are higtilig''f«' in die tide is that fiaod. aod the concomiiant bsi reroinc dtasAonx. b 
appareedy a ptoWem in today's plasde card based economies. The diferenrial advaatases and 
disadvaniages of this syJteia relative to cuhbb: systems will be dismssed after an iUustiaiive embodinem 
isprescsted. 

Fig. 22 illustrates die basic unftHBOble plastic card which is quite unique to each and every user. 
Atflgital image 940 is t^ of die user of the card. A computer, which Is hooked into die central 
accounting network, 980, depicted in Fig. 26, receives the digital image 940, and after processing it (as 
wiU be described snnoundiog Fig. 24) produces a final rendered imags which is d»eo printed out onto die 
personal cash card 950. Also depicted in Fig. 22 is a straightforward idendficaiioa oaiidag. in dm 
15 bar code 952, and optional posidonMidaU which m;qr assist in simplifirmg die scamungiol^^ 

Reader 958 depicted in Fig. 23. 

Tl» shoK story is dW die personal cash card 950 acaiaily contains a very larp ainoont of 

iofbomiian amque id dial panicular card. There are no magnetic strips involved. d»a^ d» same 
principles can certainly be Jetted to magnetic strips, such as an implaaied magnetic noise signal (sec 

20 earlier discussion on the "fiagetprinting' of magnetic strips in credit cards; here, die fingerprinting would 
be prominaii and proactive as opposed to passive). In any event, the unique infiwmation widiin the image 
on die personal cash card 950 is stored along widi die basic account information la a cemial aaawming 
network, 980. Fig, 26. The basis for unbreakable security is that during aansacaons. die ceatrai network 
need only qiKty a sinaU fiacdon of the tool iaftemation contained on die card, and never ne^ 

25 Ihe same precise infotasaioii on any two transactions. Hundreds if not diousand* or even tens of 

dwosands of mnqoe sad sectiie 'transattloB tokens' arc contained within a single personal cash card. 
Would4» pirates who we« so as to pidc off transmissions of eiflier encrypted or evea uneactypted 

tiansactions would find die infotmadon useless diercafter. This is in marked distincticn to systems which 
have a single ooBJpiw and complete 'key" (generally eticrypted) whic± needs to be acc^ 
cmirety.ovtx and over again. The personal cash card on die odier hand contains tftouaamls of separate 
and secure keys whidi cm be used once, widun milliseconds of time, dien forever ditown aw^ (as it 
were), ite ceatrai network keeps trade of die keys and knows which have been used and winch 

bavea'u 

Fig. 23 depicts what a typical poim-of-sale reading device, 95». might look like. Clearly, such a 
dcwicB woidd need to be mannfMarable at costs weU in line widi..or cheaper dian. currm ^ 
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systems, ATM system, sad cxedit csni swipen. Not depicted ia Fig. 23 are the imuxds of the opcicai 
scanning, image pracessmg, and data comouuiicatiOQs coaipoDexics, which would simply follow aomsai 
engineexii^ design meduids canytog out the funcnaos ±at arc to be described hcncefonh aod are well 
within the capability of anisaos ia these fields. The reader 958 has a aomeric ^m± pad 962 on it, 
showing chat a normai penooal idemificatioa oumber system cao be combined widi tbc ovcrai! des ign of 
diis system a d^mg one inare convemonal layer of atairity (generally after a theft of the physical card has 
Gccuned). It should also be pointed out that die use of the picmze of die user is another strong (and 
increasingly common) security feanire mtsnded to thwart after-dieft and illicit use. Functional elements 
such as the optical window, 960« are shown* mimicidng die sh^ of the card, doubling as a centering 
mechanism for the scanning. Also shown is the data liite cable 966 presumably connected eldier to a 
proprietor's central meicfaam compnter system or possibly duectly to die cea&ai network: 980, Soch a 
reader may also be attached directly to a cash register which perfoims die usual tallying of purchased 
items. Periiaps overkill on seairity would be ±c construction of die reader, 958, as a type of Faraday 
cage such diat no electronic signals, such as die raw scan of the card* can emanate from the unit. The 
15 reader 958 does need to contan, preferably, digital signal processing units which will assist in swiftly 
caicnlanng die dot product operations described hcnccfordi. It ^o should contain local read-only memory 
which stores a muliimde of spatial patterns (die oithogonal patterns) whidi will be miKzcd in die 
"recognition" steps ouiliXBd in Fig. 25 and itt discussion. As related hi Fig. 23, a consumer using die 
plasdc card merdy places dieir card on die window to pay for a oansaction- A user could choose for 
20 themselves if diey want to use a PIN number or not. Approval of die puxdiaso would presumably happen 
wittun seconds, provided dsn die signal piocessiiig steps of Fig. 25 are properiy implemented widi 
effectively parallel digital pcocessmg hardware. 

Fig. 24 takes a brief look at one way to process the raw digital image, 940, of i user Inio an 
image with more usciul infonmrion content and uniqueness. It should be cleariy pointed out that the raw 
25 digital image itself could in fact be used in the falkiwing mediods. but diat placmg in aikfitxcnal ordiogooal 
pattern into die image can significantly inciBase the overaUsysim (Oidiogonal means diat if a given 
pattern is mnKpiied by anodier orthogonal pattern, the resulting number is lero, where •nxttUiplicatkm of 
patterns' is meant ia die sense of vector dot products: dicsc arc all familiar terms and concepts m die art 
of digital hnage processing)- Ftg. 24 shows dial die computer 942 can« after intciiogating die raw image 
30 970, generate a master snowy image 972 which can be added to die raw fanage 970 lo produce a yet-more 
nniquc image which is the image drat is printed omo die a The ovcraU effect 

on the image is to •texmiize" die image. In die case of a cash card, invisibiiiry of die masttr snowy 
pattern is not as much of a recpiircmem as widi commeioai imagery, and one of the only criteria For 
keeping die master snowy image somewhat fighter Is to not obscure die Ima^ of die usff. The central 
netwoik, 980, aores d» final processed image in die record of die account of die user, and it is diis 
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uaique axul securely fcepi image whidi is tbe carrier of ibe highly secure *ihrow-away tiansaaion keys." 
This '^Tffa ge will thcttforc be "made avaii^le" to all duly coonected point-of-sak locations in die overall 
networlc. As will be seen, none of the point-of-sale locaitions ever has knowledge of ibis image, they 
merely answer queries from the central n^orfc. 

5 Fig. 25 steps through a typical transaction sequence. The figure b laid out via indentations, 

Ti^iere ±e first column are steps pei^Timed by the point-of-sale reading device 95d, die second coiuom bas 
iofonoaoon transmission steps communicated over the data line 966, and the diixd column has steps taken 
by the central network 980 which has die secuied information about die user's account and the user's 
unique penonal cash card 950. Though dtere is some parallelism possible in die implementadon of die 

10 steps, as is normally pracnced in the engineering inqslemcmaiion of such systems, the siqps are 
nevertheless laid out according to a general linear sequence of events. 

Step one of Ffg. 25 is the standard 'scanning" of a personal cash card 950 widiin the opdcal 
wbidow 960. This cm be performed using linear optical sensors which scan the window, or via a two 
dimensional opricol detector array such as a CCD. The resulting scan is digitizEd into a grey scale image 

15 and stored in an image &amB memory buffer such as a "framegrabber,* as is now amunon in the designs 
of optical imaging systems. Once die card is scanned, a fint miage processing step would probably be 
locating die four Sducial center paints. 954. and using dicse four points to guide all fuxdicr image 
processing operaions (i.e, dm four fidudals •regisier* the corresponding patterns and barcodes on die 
pensonal c»h catd). Next, die barcode ID number would be extracted using common barcode teading 

20 image procosing mediods. Generally* die user's acconms omnber would be determined *m dus step. 

Step two of Fig. 25 B dm optional ^ing in of ±c PIN number. Presumably most uscn would 
opt to have diis feature, except timse users who have a hard dme remembering such diings and who arc 
convinced diat no one will ever steal their cash card. 

Step tfaies of Fig. 25 entails connecting through a data line to the central acco u n ting network and 

23 doing Che usual conummications b^<j<ha^"ng as is commoQ in modem-based coimnumcatmns systems. A 
mote sopfalsdcated embodimem of this system would obviate the need for standard phooe lines, such as the 
use of optical fiber data Imks, but for now we can assume it is a garden varisy beUtone phone Ime and 
diat the reader 958 hasn't forgotten die phone number of die central network. 

After basic communications are established, step four shows that die pQint*of-salc location 

30 transmits die ID number found m step I, ahmg widi probably an enaypted version of \hs PIN number (ks 
ftrfH^ secority, such as using die ever more ubiquitous RSA encryption mediods), and a^iends die basic 
informaiott on die merchant who operates die point-<rf^salfi reader 958, and die amount of die requested 
iransaciioa in monetary uniis* 

Step five has die central network reading die ID number, routing die m&imation accordingly to 

35 the acmal memory locaikm of diat user's accoum, dicreafter verifymg dac PIN number and cheddng dat 
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the- accotmt balance is sufficiem to cover the trassacuoo. Along the way, the central network also accesses 
the mcxdiattt's accoam, chedcs that h is valid, and readies it for an vnictpated cxEdii. 

Step six begins wiiis the assumptioa [hat step five passed all counts. If seep five didn't, the exit 
step of sending a NOT OK back to the oKnihant is not depicted. So. if everydiing checks ont, the camal 
network generates twenty four sets of sixteen numben. where all mtmbeis ace mutually exchuive, and ia 
general, there wiU be a laige but ([oite definitdy Mte lai^ Ftg.25posifs 
die tange being 64K or 65536 mmibas. It can be any practical number, actuaay. Urns, sec one of the 
twenty four sets migbt have the numbers 23199. 54142. 11007. 2854. 61932. 32879. 38128. 48107. 
65192, 522, 55723. 27833, 19284. 39970. 19307, and 41090, for example. The next set would be 
similariy random, bttt the cumbers of set one would be off limits now. and so on through the twenty four 
sets. Thus, the central network would send (16x24x2 bytes) of numben or 768 bytes. The actual amount 
of numbers can be determined by engineering optimizaiitm of security versus transmissioa speed issues. 
These randmu numbers are actually indexes to a set of 64K universaUy a priori defined onhogonal patterns 
which arc well known to both die cenaai network and axe permanently stored in memory m all of the 
15 point-of-sale readers. As wiU be seen, a would-be thief 3 knowledge of these pattern w of no use. 

Step seven dicn ttansmits die basic 'OK to proceed" message to die reader, 958, and also sends 

die 24 sets of 16 random index numbers. 

Step dght has die reader leceiviDg and storing aUdicse numben. Then die reader, using its local 
. microprocessor and caaiam dcsigm^ high speed digital signal processing cixcuiny . steps timugh all twenty 
20 four sets of ounAcB witb the intention of deriving 24 disrinct floamig point numbers which it will scad 
back to the central network as a "one time key' against which die central network will check die veracity 
of die card*s image. The reader does dxis by first adding togedicr the sixteen patiems indexed by die 
sixteen random nnmbcn of a given act. and dien performing a common dot prodna operation between die 
resulting composite panern and die scmnedimag?B of flKc^^ TTifi dot produa gcnerams a single number 
25 (wbich for amplidiy we can catt a floaring point number). The reader steps dMUgb all twenty four sets 
in like firfiion^ generating a unique stri3« of twenty 

Step nine dnims die reader transmitting diese results back to die centz^ B^^fi^* 
Step ten dieahas diecanral networic performing a check m diesc rcmraed twenty four numbers, 
presmnably doing iia own cm same calculations on die stor^ 
30 basinttsqwnmcmoiy. Tlic mmibcrs sent by die reader can be- -normalized/ meaning dia^ 

absohue value of die collective twemy four dot piDdnos tan divided by rcclf Cits unsigned value) , so diat 
brightness scale issues are removed, m resultuig match between d:e remined values and die centr^ 
nctworic's calculated values wiU eidier be wcU widiin given tolerances if die card is vali^ 
phony or if die card is a cmde tqnoduetioii. 
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Step devea tlien 1ms the canral netwoxk sending word whedier or not the uaDsacooa was OK, snd 
Idling the cusfamer know thai they can go home widi their purchased goods. 

Step iwetve theo escpUdtiy shows how die oeiduDt's aocouit is cicdtied widi die tcaosmon 



5 As already slated, die primary advantage of dns plasnc card ii to significaiitiy tedocs band, wUch 

appai e uil y is a large cost lo conent systBSS. This syscen icdBces the possibility of fiasd only to those 
cases where die physical eanl is cidiersiDien or very careMy copied, in both of diesecases» there sdU 
remains the FEN secuncy and die user picmre secaxiiy (a known higher securiiy than hiw wage eiexks 
analyzing signatures). Attempts to copy die card must be performed dmmgh ^temporaiy thefl* of die 

10 card, and require photo-qualiiy copying devices, not sin^e magnetic card swipers. The system is founded 
upon a modem 24 hour highly linked data network, nilcit monitoring of tiansaciions does the monKoring 
party no ose wb^her the tcansniisstona are encrypted or not. 

It will be ^ipeedated diat the foregoing ipproadi to increasing du secnriiy of iransaciiens 
involving credit and ddiit card systems is readily extended to any photograph-based idnirififaTifm system. 

15 Moreover, die pxinci(te of the inesent technology may he ^plied m detect alteiadon of photo ID 
docuinents. and to gcaerally the crmfidcmx and securi t y of such sjrsGems. In this regard, 

reference is made to Fig. 28, which depicts aphotcKED card en- docoment 1000 which may be, for 
ezanpLe, a passpon, visa, permanent resideot card ("green card*), driver's license, crectit caid, 
government employee identificatioa^ or a private indtstiy idendfication badge. For convenience, such 

20 phott>gzaph-based identificatioa documoits will be coUcoxveiy referred to as photo ID documents. 

The photo ID rincmnent mnlnriRs a phatagEa{& 1010 diai is attached to die docomeni 1000. 
Pnsjcd, hnman-ieadable infonnadoii 1012 is incorpocated in die docmneat 1000, adjacent to die 
phomgraph 1010. Machine readable information, sudi as that Imown as "bar code" nu^ also be included 
adjacent to the photograph. 

25 Generally, the photo ID docnmem is conscracied so that tampering with the document (for 

exanqiLet swapping the odgmal photograph widi anodier) dxmld cause noticeable damage to the card. 
Neverdidess, skilled forgerets are able m ddier aher otisong doconcnts or mamifaant e fEandnlent photo 
ID doommms in a manner that is extremely difficult to detect. 

As mxed above, die present technology the security associated with die use of photo ID 

30 documents by siqjplementing the photographic image with encoded mformacion (which informanon may or 
may not be visually percept&le), ^lerdtij fadiiianng the corrriarion of the photographic image widi other 
.infoimaiion coDceming the person, such as the printed information 1012 appearing on the document 1000. 

In one embodimem. die photograph 1010 may be produced £iom a raw digital image to which is 
35 added a master snowy image as described above in conneeiioo with Figs. 22-24. The above^lescxibed 
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centzai occworic and point-of-sale reading device (wfa^ device, in die preaem embodiment, may be 
considexed as a point-of'esiiy or poim-of-ucurity photo ID reading device), iKuid esseuiaily cany one die 
saoK processing s described with dial embodinient, including die cenoaf oenvoric generation of uniqae 
numbeia to serve as indices to a set of defmed onbogonal pattezns, the associated dot prodoa operation 
5 carried out by die leader, and the oomparison with a similar operation canied out by die central aerwork. 
If the nwnbeis generated torn die dot product opcranon carri«i out by die reader and die central oowoA 
match, in diis embodiment, die network sends the OK to the reader, indicanng a Jegiumate or uoaitezed 
photo ID document. 

In anodier embodiment, die phoiograpfa oomponcnt lOIO of the identilicaiion docamcDC tOOO may 
10 be digitized and processed so diat die photographic image that is ircorporated into die photo ID documem 
1000 coiresponds to die *dismbutable signal' as defined above. In diis instance, dterefore, the photogr^b 
mdudes a composite, embeilcd code signal, imperceptible to a viewer, but carrying an N-bii identification 
code. It will be appreciated diat die idcndllcarion code can be extracted from ±e photo using airy of die 
decoding cccfanicpics described above, and employing cidier nniveisai or custom codes, depending upon die 

15 level of security sou^ 

It will be appreciated thai die infotmation encoded into die photograph may correlate to, or be 
redundant widi, die readable in&nnari£m 1012 appearing on die doc um ent Accordingly, sudi a document 
could be authenticated by placing die photo ID document on a sca^ 

at a passport or visa coniroi point. Hie local computer, wfaidi m^ be provided widi die universal code 
20 for extracting die identificanon information, displays die extracted infonnadon on die locd computer 

screen so that the cperawr is able to confirm die correlation between die encoded information and die 

readable infonnadon 1012 canied on die document. 

It wiU be apptcciaiod diat die information encoded widi die photograph need not necessarily 

coirelaie widi odier hribrmaiion cm an identification documem. For example, d» scanning system may 
25 need only to cwifim die existence of die idenrifiationco^ 

or "no go* indicatiott of whcdier die photograph has beca tampered widL It will also be iQ?pieciated dial 

dm local computer, using an encrypted digital commumcatinns line, could send a packet tif m&imadon .to a 

ctttral verificaiiim fedliiy, *hicfa dietcafter lenmis an encrypted 'go" or "no go' fadication- 
In another eodiotfimcnt, it is contemplated diat die 
30 mav be a robust digital image of biomeiric dan, such as a fingeiprim: of dm card bearer, which image, 

after scanning and display, may be employed for comparison widi die actual fingerprint of die bearer in 

very high securi^ access points where on-die-spot fingerprint leoogniiion systems (or rcdnal scans, etc.) 

are employed. 

It WiU be appreciated diat me information embedded in die pbotograpfa need not be visuaUy hidden 
35 or sieganographicaUy embedded. For example, die photograph incorporated inw die idemification 
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may be a composite of an iisage of the indh/iduai and one-, or two-<limeasional bar codes. The bar oode 
infonziation would be subjca to convemional optical scanning tecfaniques (including nueznal cross checks) 
so thai the mfannacoa derived from ibe code may be compared* for exan^ie, to the infonnadon ptiitfed 
on the identificaiioa documeat. 

5 it Is also conteinpiated thai the photogtapha of ID documents coneaiiy in tise may be processed so 

thar infomiation coErelated to the individnal whose image appears in the ptaoiogx^ may be embedded. In 
this regard, die reader's anention is diieoed to the foregomg portion of this description emitied TJse in 
Priming. Paper, Documents, Flasxtc-Coated Ideotificatioa Cards, and Other Material Where Global 
Embedded Codes Can Be Impiimed,* wherein diere is described nnmerous approaches to modulation of 

10 physical media diat may be treated as 'signals* ameoaUc to application of the present technology 
ptincipies. 

WFTWQff K: UNKING MFTH OD USING INFORMATION 
FMRgnPED IN DATA OBJECTS THAT HAVE INHER ENT NOISE 

15 The diagram of Fig. 27 iUustrates the aspeet of the technology that provides a nscwotk linl^ig 

Kffldxod using information embedded in data objects dia In cm sense, rfiis aspect is a 

network narigaiion system and. more broadly, a massively distributed indexing system diai embeds 
addresses and t^rfirina dircaiy within data objects themselves. As ODced, dais aspect is paiticulariy well- 
sd^ied tor establishing hot Unks widj pages presented on die World Wide Web (WWW), A given daia 

20 object effective^ contains both a gr^cal representation and embedded URL address. 

As in previous embodiments, diis embedding is carried cut so that die added address infonnadon 
docs not affect the cote value of the object so far as the creator and audience are canccmcd. As a 
consequence of snch embedding, only one class of data (Ajecis is present rather Uian die two classes (data 
objea and discxete header file) thai are aneiidani with traditional WWW 1^ The advantages of reducing 

25 a hot-Linked dam objeei to a single class were mentioned above, and are elaborated upon below, hione 
i>m> v.H Wnt ftf thft igghaology. the Worid Wide Web is used as a pre-existing hot link based network. The 
common appaja i us of this system is networked computers and conqmier cnonitors displ^ing the results of 
intP ra fftifOPf when connected to die web. This end}odimeQt of die technology comemplates 
stcganographicaliy embedding URL or odier address-type information direcUy into images, videos, audio, 

30 and odicr fonas of data objects diat arc prtscmed to a web site visitor, and which have 'gray scale" or 
•continuous tooes* or •gtadatitms" and, coDseq[ucctiy, iidictcnt noise. As noted above, there arc a variety 
of ways to ttdht bsie steganographic unplementatians. aU of wtiidi could be employed m accordance 

widi the present techoology. 

With particular lefetence to Fig. 27» images, <iu83i<ontinuons tone graphics, mnltiinedia video 
35 and audio data arc caricnriy die basic buikling blocks of many sitc^ 1002. 1004 on the World Wide Web. 
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Such data will he heieaiter ooUectiveiy lefemsd to as creative data files or data objects. For iUustiative 
purposes, a comiauawonc graphic data objea 1006 (diamond ring with background) is depicted m Fig. 
27. 

Web site tools - both those that develop web sites 1008 and dnse chat idlow browsing them 1010 - 
5 loudneiy deal with the various file fonnats in which these data objects aie padcaged. It is already consnoa 
to distribute and disseizduate diese data objects 1006 as widely as possible, often widi the hope on a 
creator's part to seil the products represented by the objects or to advertise creative services (e.g., an 
excjupiary phatograph, widi an 800 phone number displayed widiin it, promoting a photographer's skills 
and service). Using die methods of this technology, individuals and organizanons who crcaic and 
IQ (fi<H ?^Hiati> such data objects can embed an address link Uiat leads rig^ hack to their own node on a 
Bciwork. dicir own site on die WWW. 

A user ai one site 1004 needs merely co point and dick at the displayed object 1006. Use 
software 1010 idpmifigs die object as a hot link objea. The software reads the URL address thac is 
embedded widun die object and routes tbe user to die linked web site 1002, just as if die user had used a 
15 oonvcnnoaai web link. Tlwi linked site 1002 is the home page or network node of die creator of the ol^ 
1006, which creamr may be a manufecrurcr. The user at die first site 1004 is dicn presented widi, for 
fyatppie, an otdcr form for purchasing the prodiia represented by the object 1006. 

It will be appreciated (hat the creatoK of objects 1006 having embedded URL atfri r cssfts or indices 
(which objects may be referred to as 'hot olqects") and the mannfacniien hoping to advcnise dicir goods 
20 and services can now spread dieir creaaive content like dandelion seeds in die wind across the WWW. 
know^ diac fm h"^^ widiin those seeds are links bade to dieir own home page. 

It is amiemplalBd that die object 1006 may include a visible icon 1012 (such as die exemplary 
TIO'abbrevianonshownittFig.27)inccaporatcdaspartofti^ The kon or other subtle hslida 

would apprise the user that the object is a hot object, carrying die embedded URL address or other 
25 hitomadott diat is aocessibic via die software 1010. 

Any humaniJexcepoble indidmn (e.g., a short musical tone) can serve the purpose of apprising - 
dK user of die hot object It is comeaqjlated, however, dial no sudi nididum is lequii^ A user's tiial- 
and-error approach to clicking on a data object having no embedded address wiU merely cause the software 
to look for, but not find, the URL address. 

TTm amomadon process Inherent in the use of dns aspea of die tcdmology is very advant^eous. 
Web software and wdi site development tools simply need to recognize dus new dass of embedded hot 
links (hot objects), operating on diem in red tii^^ Conventtond hot links can be modified ami 

supplemented aunpiy by "uploading' a hot objea into a web site repository, never requiring a web site 
progranuoff to do a thhig odier dian basic monitoring of traffic. 



30 



SUBSTITUTE SHEET (RULE 26) 



. PCT/US97/08351 . 

WO 97/43736 



76' 



A metbod of unplementing die above described funciioo^ of the present tedmology ge&enily 
iovoives the stqps of (1) creating a set of standards by which URL addresses are stegaoographically 
embedded within images, video, audio, and other fbims of data objects; and (2) designmg web sice 
development tools and web software such diat they recognize this sew type of dau object (the hot objea), 
5 die tools being designed such tbat when the objects axe presemed to a user and diat user points and clicks 
on such an objea, the user's software ksiows bow to read or decode the steganographic mfbcxnatioa and 
route [he user to the decoded URL address. 

The fbregoiBg portiotis of this desctipiioa detaOed a steganogiaphic implementatioii (see, 
generally. Fig. 2 and die text associated dterewith) that is readily adapted to impictseot die present 

10 tedmology. In this regard, the odicrwisc convcntkmai site (fevdopmcnt too] 1008 is mhanrrri la i n c lu de , 
for exaiaple, the capability to encode a bit-mapped image file with an identification code (URL address, 
for example} aooording to the present technology. In the present embodiment, it is contenqsiated that the 
commercial or transaction based hot objects may be steganographicaUy embedded with URL addiessea (or 
odier mfbnnadcHi) using any of the univeEsai codes described above. 

15 Tbe foregoing ponioos of diis descxipdoa also detailed a technique for reading or decoding 

ategatmgraphically embedded mfbrmaticai (see, generaUy, Fig. 3 and tbe text associated ttierevidi) diat is 
readily adapted to isxpicmeot die present technology. In this regard, the otherwise conventional user 
software 2010 is cnhaxKed to indade» for example* dw capability to analyze encoded bit-iiiapped Slcs and 
extract rbe identification code (URL address, for example). 

20 While an miKtrative implementation for steganograf^caily embedding information on a data 

object has been described, one of ordinary skill will appreciate diat any one of the multitude of available 
stegamgraphic techniques may be employed to cany out the function of ttie preseat embodiment. 

It will be appreciated tbat the piesenc embodlmBOt provides an inuoediarc and comznon sense 
mecfaanismwhereby someof die foadamenial building bloclcs of die WWW, namdy nm^es and sounds 

25 can also become hot links to other web sites. Also, die iffo g n mmiDg of such hot objects can become fidly 
automated merety duoogh die distrxbcdoa and availability of unages and audio. No real web site 
programmbg is required. The piesenc embodiment provides for die comn^rdaJ use of d» WWW in such 
a way diat tjoa-progranmen can easily spread their message nserely by cteaiing and distributing creative 
content (herein, hot'objects). As noted, one can also traasiiion web based hot links tiiemselvcs torn a 

30 more arcaae text based interface to a more tiasural image based mterface. 

Eacapsuiated Hot Ltnic File Formal 

As noted above, once steganographic otfchods of hoc link navigaiion take hold, then, as sew file 
formau and traasnussion protocols develop, more traditional metbods of "header-based' information 
35 arr ^ ttmi^ t tf cffli cohaBce diG basic approach built by a sieganogxqihio-based systeoi. One way m begin 
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euending the stegao^i^ibie based lioc lint aetbod back imo die niore traditional beader-based mediod \a 
to defiae a new class of Se fonnac wfaidi could efiiBctiveiy become dtt standard dass used b network 
navlgatioii systems. It will be seen ihac objects beyond Izna^, audio and the like con now become "hoc 
objects', including cbs files, indexed graphic files, computer prognms, and the Iflte. 

The encapsulated hoc link (EHL) file fionnit simpiy is a small shdl placed around a large range of 
pre-exisilog file tomais. The EHL header tnibnxiaiioa cakes only ihc fiist N byiss of a file, followed by a 
M and e;uict file hi any kind of industry standard formal. The EHL super-header sserely attaches die 
correct flic type, and the URL address or odur mdex iniormaticn assadadng diat objea to other nodes on 
a networic or other daiabases on a necwork. 

It is possible that the EHL format could be the mediod by wtuch the steganographic mediods are 
slowly replaced (but probably nerer oon^eteiy). The slowness pays homage to the kiea diat file format 
staodaids often take much longer to create. impiancDt, and get everybody to acmally use (if ac aO). 
Again, the niea U that an £HL-like format and sysian built around it would bootstrap onto a system sentp 
based on steganographic methods. 



Self Earaetin pi Web Qbiecs 

Cenenily speaking, three classes of data can be steganographicaUy e m h e drifd in an object: a 
number (e.g. a saiai or identification number, encoded ui binary), an alphanumeric message (e.g. a human 
readable name or tdephtmc number, encoded in ASCn or a reduced bit code), or ccamputer instructions 
20 (e.g. JAVA or HTML tastrucriocs). The embedded URLs and the like detailed above begin to cxploie diis 
third class, but a more detailed exposidoa of the possibifitles may be helpflil. 

Consider a typical wd) page, shown m Fig. 27A. It may be viewed as mchxling three bask: 
components: images 01 • #6)i text, and layout. 

Applkant's techoclogy can be used to consolidate Uiis uiibrmation into a self-cxmiciing object, 
25 and regenerate die vfdb page from this object. 

In accordance wi± diis example, Fig. 27B shows the images of the Fig, 27A w^ page fitted 
together imo a sin^e RGB mosakzd imgc. A user can perform this operation manuaUy using existing 
image processing programs, snch as Adobe's Photoshop software, or ihc qpcratwn can be automated by a 
suitable software program. 
30 . Between certain of dw image tiles mtiw Fig- 27B mosaic are en^ar^ 

hatching). 

This mosaxced image is then stegamgxaphically encoded to enAcd the layoui instnictiotjs (e.g. 
HTML) and the web text therein. In the cn^ty areas the encodiflg gain can be mai i niiT Rri since djcrc 
is no image data to cociupt. The encoded, mosaiced image is dien JPEG compressed to form a self 
35 exnaciing web page object. 
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(JPEG comptsssion is used in this example as a lingua franca of image repnsencations. Anotiier 
such candidate is the GIF fOe fbnnac. Sudi fbnxuts are supponed by a vahety of software toois and 
languages, maidng than well suited as "^coimRoa canicrs* of embedded inibnnauax Other image 
leptescncttions can of course be used,) 

5 These objects can be exchanged as any oUier JPEG images. When die JPEG file is opened, t 

suitably piognmmed compmer can detect the presence of the embedded infoimaiiOD and extract the l^ttt 
data and text. Among «her infionnaiioo, the layout data spedfies where die usages forming die mosaic 
axe to be located m die &oai web page. The compmer can follow the embedded HTML Lutnictiofls to 
leoeate die original web pa^« complete with graphics, text, and links to odier URLs. 

10 If die self extracting web page abject is viewed by a conventional JPEG viewer, it does not self- 

extract. However, tbe user will see the logos and artwork associated widi die web (with noise4ike 
"groining" between certain of the images). Artisans will recognize that diis is in stark contrast to viewing 
of other compitsscd data objects (e.g. PKZIP files and self extracting text archives) which typically appear 
totally tmintdligible unless fully extracted, 

15 (Hie foregomg advantages caa largely be achieved by piadng die web page text and layout 

iostnictions is a header file associated widi a. JPEG-compiessed mosaiced image file. However, industry 
scandardizazicm of die header formats needed to make such systems piacdcai appeats difficult, if not 
itnpossibte.) 



20 Paiettes of SteganDgmiAically Encoded Images 

Once web images with embedded URL tnfoimation become widespread, such web images can be 
collected into •palettes* and presecied lo users as high level navigation tools. Navigation is effected by 
clicking on sucb hnages (e.g. logos for different web pages) nuher tbaa cliddng on lexxual web page 
naniK. A suitably programmed computer caa decode the embedded URL information from die selected 

25 image, and establish die lequested connection. 

In one embodinient« self^extracdon al its above-described web page objects automatically 
generates dumbnail uoages corresponding co die extracted pages (e.g. iqsessntative logos), whidi are 
dien stored in a subdiiectoiy in the compute's file system dedicated to coUecui^ such thumbnails, to eacb 
sudi dimrfmail is embedded a URL. such as the URL of the extracted page or die URL of the sire from 

30 which die self-extracting object was obtained. This subdirecioiy can be accessed to display a palette of 
naviganooal i*ii"«hiiaiu for selection by die user. 

Spgrife gtarrmle of a Cotnocter Svstem Lmked bv Using Infannarion in Data Qbjet^t^ 

Hie present mvention allows a digital watermark to be embedded direcdy iuo photographs, video, 
35 computer images^ audio, and odier forms of creative property. The range of applicadons is amazhigly 
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and tftduda tbisgs like video and movie comeot waiensaridng for proof of ownersiup and crackisg, 
watermarking nmsic and audio oooicQt, even texnirmg physical maxenals sucb as amo pans. As discussed 
above, dus watermark is in^erceptibie to die eye, and imperceptible Co the ear, but a computer analysis 
can tead the watermark and discover a message it conies. The watermark is repeated thronghout die 
5 property. It is rdtmu and typically sorvives muittpie genedciotts of copying, modiiication, printing, 
scanning, and compressioa. A watermark may cany a copyright nonce, a unique serial monber. a 
Transscdon ID, as well as other appikation ^edfic data. 

ConcepmaiIy» the example bekiw is made wiili respect lo a watermark embedded by 
'PictureMarc'' which illustrates the idea by carrying a unique creator id, and a set of image 

10 attributes. This creamr id coiresponds to complete contaa information available through an on- 
line service, MaicCentre. Technically, the watennaric is 128 bits long and has a data payioad of 
76 bits, although die invention is not resthaed to any particular watennark size. In tiiis example, 
it is short so that it can be rcpealsd many times through an image, making it possible to find die 
message even frons a portion of the original image. Other approaches can include data such as an 

15 image serial number or a Transaction ID m the watermark message. It is also possible to add a 
person's name or odier such data (to a computa; it b all data)» but one application is focused on 
embedding a more structured message. First, to accommodate a text string, there would be a 
tendency to push for longer messages. Long messages rsuh in not being able to fit very many 
copies of tbe message in an image» and thus the watermark becomes less robust Also, Just 

20 putting a name in an image is much like the credit line in a magazine: A. Jones tells who the 
person is, but not how to contact the person. With a watermark system available on line, a user 
can get complem and up to date contact details from each Creator ID dirough die on-line service. 
Acosrdingly, it is preferable to associate persistent digital infbimation with an image. The cm 
software has been designed to enable working widi vendon or large customers who may want to 

25 develop qiecific watermark message types for special applications such as embedding the fikn 
speed and exposure in Ac image, or serializing images in a large collecdon for internal tracking 
and management. 

One approach to implementing diis embodiment involves TictureMarc", which includes a 
writer porti<m to embed watermarks and a reader portton to find diem and read dieir contents. 
30 This bundle is integrated into Adobe Photoshop® 4.0 (Adobe Photoshop is a trademark of Adobe 
Systema, Inc.) as a phigin extension, and fiom the end-user's pcrspcclive is ftee. A free reader 
will also be avmTable dirough a web site. PictureMarc for Photoshop is supported on Apple 
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Macintosh and N^icrosoft Windows platfonns. PictureMarc supports embedding a Creator ID and 
Image afcributts in an image. PicmreMarc also has the ability to cany a unique Image ID or even 
a Transaction ID in the wacennark. These formats will support tracking images bc«h to a 
coUecdon, and to the person and terms that were in place when the image sold. The watermark 
5 reader will accept and read all watmnailc fonnais. 

As an example* the first step is to read an image fiom a CD-ROM, download a fiJe to a 
local computer over the Internet or convert a hardcopy image, such as a photograph, mto digital 
by ^rpnn ^^s the unagc with any of the common image scanners.. Then start an image 
manipulation program that sports watermaricing, and load the image file from a file selection 
10 menu. Select the menu item to add a watermark, and save die watermarked image into a file. 

To recover a watermarked image from a digital image on, say» a PC, just as in the case of 
embeddmg a watsennark, convert the image to a suitable computer file format, scanning it in if 
needed. Then scat eirficr an image manipularion tool supporting watermarking, or start a stand- 
alone reader. Now open the image. In the case of a preferred embodiment image manipulation 

15 toot the tool will autDmatically check for a watermark when the digital image file is opened, 
copied to the system clipboard or the image is scanned into the computer. If present, a copyright 
symbol is added to die title or status bar, or some other signal is provided to the user. By 
selecting a read watermark menu option, a user can discover the contents of die waiennaik- From 
the results dialog, a user can dick on the Web Lookup button to retrieve complete comaa 

20 information from MarcCeatrc, a network WWW ra-linc service. The application can be carried 
out widi at least two kinds of watermarks. The first is a public watermark, whkh can be read fay 
any suitably configured reader. This is die type of watermark supported by PicturcMarc-Its 
purpose is communication. The second is a private watermaric Private watermarks are associated 
widi a secret key used at die tune the watermark is embedded in an image. Only die person widi 

25 die secret key can find and read a private waternsarifi- Private watermarics are used in proof of 
ownership applications. Bodi a public and a private watermark can coesdst in an image. 

If a user attempts to read a watermark and none is dicrc, die reader portion will simply 
report diat a watennark was not found m the image. If a user attempts to watermark an ima^ 
already watermarked, the user will be informed dicre is aheady a watermark in Uie hna«e, and 
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will be prevented fiom adding another watermark. In creating a composite image, the user is 
told about the presence of a watermark in each watermarked original (assuming the use of a tool 
supporting warennarks). This addresses die first order problem of communicatmg copyright The 
watermark will not prcvwit crcadng a composite work, in that a wateraxarked image, or a portion 

5 thereof, can be used as an element of a composite image. However, if the composite image 
contains muitiple watermarked elements, the reader portion will tend to read the watermark from 
the most dominant piece. However, if the user suspects a component to be watermarked, or is 
simply inmsted io the source of an image element, the user can check for the watermark in that 
element by highlighting the area in question, and invoking the read watermark fimction. This 

10 provides a tool to inspect work to discover sources of individual elements. Copyright hierarchies 
in composited works and parmering with companies addressing rights man^ement can all be 
handled in such a network application of a watermark. 



Preferably, applications which mclude watermarking, such as Photoshop 4.0, support 
automatic drfection ("autodctect"). This means that whenever an image is opened or scanned into 
15 the application, a quick check is done to deteimine if a watennaric is present If it is present, a 
visual indication, such as a copyright symbol, is added to die tide or status bar. 

The watermark placed by PictureMarc is a 128 bits long, with a data pay toad of 76 bits. 
The remaining bits are dedicated to header and control information, and error detection and 
correction. HowevCT, the bandwidth can be increased and consequently the data payload can also 
20 be increased. The detect process looks for the presence of a watermaik, \Aich completes in a 
couple of seconds, usuaUy under 3 seconds regardless of image size, and the typical time to 
perform a full read watermark operation is under 15 seconds. 

PictureMarc supports all the file formats of die host application, since it works widi pnteis 
la each of die major color schemes (CMYK, RGB, LAB, Grayscale). In the case of PictureMaic 
25 for Photoshop, all of the Photoshop file formats are supported. 

Note diat the watermark itself carries version and message type information, much Uke a 
communications message. PicnireMarc has been designed to support upward compatibility. Any 
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suitably configured reader will be able to read previous versions of the watctmark. If a reader 
finds a newer version watermark which it do£d not support, die user will be notified and told to 
download the most recent reader. 

In the present application of PictureMarc, an image goieraily needs to be at least 256x256 
5 pixels to cany a watermark. Often a watermark can be found in patches as small as 100 pixels 
square. Arbitrarily large images can be watermarked, constrained only by available memory and 
disk space. The waannark does not add data to the image file, since k only makes very small 
changes to the luminance of pixels. This can change the size of a compressed image by a few 
pendent, but die size of uncompressed images, such as bitmaps (.BMPs), are unchanged. While a 
10 watermark survives compression, to watermark an already compressed image» it is preferable to 
either woric fiom the original, or uncompress, watermark, and recompcess the image. Futu^ 
versions will likely support adding watermazks dorectty to the JPEG compressed file. 

KcturcMarc suppwts the automation features provided by Adobe Photoshop 4.0. This 
means that repetitive watermarking operations can be automated now. In other ^plications, 
15 support can be provided for unanendcd batch processing of images, aimed to support the needs of 
image distributors, especially those companies lookmg (o watermark issues as they are 
disoributed with unique Transition IDs. 

As to the use of a network on line service, an unage creator or distributor subscribes to 
the MarcCentre, on-line locator service. Subscribing includes providing a set of contact 

20 information to be made available when cme of a user's watermarked images is found. Contact 
details faiclude name, address, phone number, speciahy, mttimediaxy (e.g., organizatioQ, stock 
agency, repieseatative. etc.) representing the cteator^s work and so on. When subscribing, the 
user is givca a uflique Creator ID. The user provides this Creator ID to a copy of PicmreMarc, 
(me time when first using the software-based system. This Creates ED links all of the user's 

2S hff atgg^ to gcilitate contact details. Information can be queried via the Internet, or via a fax-back 
service. Querying is preferably free. 
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A subscriptioQ to M^icCentre is priced to include unlimited access to contact information 
by anyone finding watennaikcd images. A Creator CD in the image leverages the power of a 
network accessible database, and lets the image creator communicate much more infonnation than 
could fit in die image. In essence, we are creating a copyright communication system. 
5 Watennartc detection in programs such as Photoshop is automatic* The Creator ID in the image, 
along widi die image attributes, uniquely identify the image creator or distributx>r. The contact 
infbnnatioQ conesponding to die Creator ID is always up to date; no matter how long ago a 
watermarked image was distributed. To make this copyright communicatioo system woric, we 
need unique ids which correspond to die appropriate im^e creator contact information. 

10 If diere is no network connection for PictureMarc, an equivalent approach can be used: 

contact infermation can be communicated via a fiax-back service. This can be carried out by 
punching in the Creator ID on a touch-tone keypad widi telephony, and die resulting contact 
details for that image creator will be sent by &x. 

Contact details will be stored on a robust server being hosted by an [ntcmct provider. 
13 The Web site is responsible for maintaining the locator service and ensuring diat infonnation is 
available. Individual image creators are responsible for maintaining their own contact details, 
keeping diem current as infonnation changes. This ensures that all of the images distributed point 
to up-to-date information about the creator and the creator's work. 

Data in die Web site repository is stored in a way thai when a user enter contact details, 
20 the user has control over which pieces of infonnation are viewable by viewers. Thus, for 

example, die user can designate whedier to exclude his or her address, phone number, and other 
specific information. 

It is hdpfid for users to receive regular reports, generated by the Web page site software, 
about die number of people querying a specific user's infonnaiion. These can be sent to die user 
25 via e-mafl or 6x, to help communicate die aaivity conccmmg watcimarkcd unages. 
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Before turning to a specific embodiment of the present iaveixuon, h is tmportanc to stress 
the gCDeml applicability of embedding a watennark, then rcsdiJig it to obtain data, aad then using 
the data to link to a database-preferabty via a computer network. This is applicable to aii media 
for which diis technology is useful. This inclcdes watcnnarking a photograph or a series of 
5 watermarks in a series of images (e.g., television, video), in sound or other anaksg signals (e.g., 
recordkgs, cellular telephone or other broadcasts), and other applicadons indicated herein. In any 
of these appHcations* while the watennark can contain self-ideatifying data, a centiaiized 
repository of watennark data &diicates broad, efTicient usage. 

Thus, the following specific description of a preferred embodiment of a ceno^ repository, 
10 accessible by a network, is intended to be iilustmive of the same kind of an approach for other 
applications and media suitable for watennaridng. And it is to be understood that while the 
following discussion is made in the context of image processing, the same general approach can 
be used for handling watermarks in acoustic or other applications. Also, while the Adc^ 
Photoshop application is used to illustrate the generally applicable concepts, it should be 
15 appreciated that the present invention can be used in connection with other such products fiom 
other vendors, as well as in stand alone form. 

Fig. 43 illustrates an application of the invention using such an application as 
Adobe Photoshop 4,0, equipped with a plug in version of the digital waterrnarking technology 
discussed herein and illustrated as 'TictureMarc," running on a digital electrical computer. The 
20 computer nmnlng the Adobe application is adapted for communication over a network (such as 
the WWW) witii another digital electrical computer, running Ae MarcCentre Web Page computer 
program. 

At step 1 of Fig. 43 a creator of a digitized photographic image, the "user" of the 
Adobe Photoshop, commences by obtaining a Creator ID. Such data is created and stored in a 
25 central repository designated in Fig. 43 as MarcCentre Locator Service^ which is connectable to 
PictureMarc via Internet or fex. A: step 2 of Fig. 43, die user creates and embeds a watermark b 
the digital image* At step 3 of Fig. 43, that image can be stored and/or printed, possibly 
subsequent to distribution over a computer system such as the Internet or WWW. When the 
image bearing the watermark is used in digital form, at stsp 4 of Fig, 43, Aat image is examined 
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for a watennark by a computer running the PictureMarc compmer program. Preferably, the 
examination is conducted automancaily (rather than selectively) upon opening an image, e.g^ 
upon locating the image on a computo* screen ciipboanL If a watemiark is detected, at step 5 of 
Fig. 43» PictureMarc communicates with the MarcCentre Locator Service to look up creator 
5 information and present a WWW page to the user. 

The means by which a user obtains a Creator ED in step 1 of Fig. 43 is elaborated 
in Figs. 44, 45, and 46, which represent screen images of MarcCoitre, a Web Page locatable on 
the WWW. Introductory infbnnation is set forth in Fig. 44. Note the mention of a single mouse 
clicL This feature refers to a hot huttna, such as a copyright symbol in a title bar of a screen, 
10 which permits someone using the Web Page to obtain iofonnation about the image that was pre- 
specified by the creator. (A hot button (pointer to information stored in a MarcCentre database, 
which can m mm have a pointer to another database, such as that accessed by means of the 
network or WWW.) 

Figs. 4S and 46 show sequendai screens of a "form** to be completed by computer 
15 entries. This infonnation is stored in a MaxcCcntrc database. Note too that the mfonnation 
entered to complete the form include a fee, and the fee account for the user is also stored in a 
MaicCentre database. After the form has been completed and stored and the fee has been 
secured, a Creator ID is issued by the MarcCentre service. 

lie means by which a user with a Creator ED embeds a registered waiermaik in 
20 step 2 of Fig. 43 is elaboiated m Figs. 47, 48, and 49. The Photoshop software uses a filter to 
call up the "Digimarc" PictureMarc software and select options to embed a watermark. Bolded 
baxs on the screen of Fig. 47 illustrate a selection being made for a digital image in die 
background. Other mfoimation on die screen is typical for a windows applkation, the Adobe 
Photoshop application, and or generally known mforraation about digital im^es, e.g., 100% 
25 shows tiie scale of the nnage, -RGB" indicates the image fonnat, etc. 

Fig. 48 illustrates a "Diafag Box" used in associating information in connection 
with the embedding of the watennark. In a portion of the screen, the Copyright Infonnation is 
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spccified, induding the Creator ID, the Type of Use (cither Restricted or Royalty FceeX Adult 
Content, and a Peisonaiize option. A demonstration ID Tor use by those who do not have a 
leistered Creator ID is "PictureMaic Dema* The Adult Content feature permits embedding in 
the watennaik data, preferably content^indicative data. Here the data is used for identifymg that 

5 the photograph contains adult content, and thus a subsequent use of the photogtaph can be 

testricced. However, an aitemativc is to include other data or odier can1en^ldaltj^ying data, such 
as cataloging infisnnadon, in the watennadc itself. Cataloging infbnnati<»] pennits subject m^er 
searchmg in a coUeaion of photographs. Another feanice shown in Fig. 48 is a slidabie scale for 
adjusting the intensity of the watermark. The more visible die watcrmart the more durable die 

10 watermark is. Watermark durability (visibility) is also indicated by a number diown on the 
screen. Once die screen has been completed, if it is "0K^ die user can click on this button and 
diexeby embed the wat^maik. Otherwise, the user can select to cancel and thereby abort the 
process diat would have led to the wacennaxkmg. 



Selecting the Personalize option of Fig. 48 calls die screen illustrated in Fig. 49, 
15 whidi is used to obtain a pCTSOoalizei registered Creator ID. Aiter the Creator ID is entered, 
selection of die Register option launches a WWW browser directed by die URL address shown 
entmd on die screen to the MaicCentre registzaxion page. 



In a preferred embodiment of step 4 of Fig. 43, each time an image is opened, 
scanned in, or copied to the cUpboard by any application, the image is automatically tested for a 

20 watermark, as JUustrated in Fig. 50. This "autodetect" approach generates a signal indicative of 
die lesult of the test, for example, one or more copyri^ symbols shown in the title page of die 
soeen in Fig. 50. Aitomatic checking for a watermark takes less computer rime dian the 
atemative of selectively reading die watennark, which is illustrated in Figs. 51 and 52. 
However, the sdective apprtjach is less comprehensive in monitDring the use of watermarked 

25 images, which is disadvantageous if the application is configured to automatically report die 
detection back to die MarcCenire. In Fig. 30, die copyright symbol in die tide area of die 
photograph is die previously mentioned hot button. The copyright symbol in die lower left of die 
screen can alternatively or also be a hot buttim. 
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Figs. S 1 and S2 elaborate the alternative approach to step 4 of Ftg. 43 in which 
an anage is opened and examined For a watennaiiL Preferably, each time an image is opened, 
there is an automatic check for a waiermak. In any case, Fig. S I is similar to the screen of Fig. 
47, except that a selection b shown to read for a watermark. 

5 Fig. 52 shows a screen that is be produced in response to the reading a 

watermaric. This screen provides the Creator ID of the owner and / or distributor of this image, 
as well as whether use of this im^ is restrictEd or royalty free. Note the Wefa Lookup button, 
which can be selected to launch a WWW browser directed by the URL address shown entered on 
the screen to the MarcCentre Web Page, and obtains the data m the corresponding MarcCentre 

10 database corresponding to the Creator ID. 

Fig. 53 illu^rates a screen produced &om selecting the Web Lookup feature of 
Fig. 53* This screen provides the contact inforraarion for the image cieator, including phooe 
number, E-mail address and WA links. Note the selectable buttons at 4e bottom of the screen 
illustrated in Fig. 53, mciudmg buttcms that cake die user to promotional o&n, allow the user to 
15 download additional softwace, subscribe to the MarcCentre service, update contact information, 
search for a creator or provkie feedback on the MarcCentre service. Selecting the Creator Search 
option leads to the screen shown in Fig, 54 This self-explanatory screen enables a user to obtain 
contact infbnnation about a particular creator by entering die Creator ID. 

In a variadcHi on the above-described approach, extended information b embedded 
20 in die watennark and subscquentiy detected, as respectively iDustraicd in Figs/ 55 and 56. Such 
a feature would be inserted as an additional filter in Fig. 47. In Fig. 55., die dialog box permits, 
mputting an Organization ID. an Item ID, and a Transaction ID. An Organization ID identifies 
an organization of odier intermediary representing die creates and/or authorized for distributing 
the photograph and licensuig rights Aereto, such as a photo stock house. An Item ID identifies 
23 die photograph in a collection, such as a CD ROM. A Transaction ID identifies die specific 
transaction tiiat is audiorizcd (e.g., publication one time only in a newspaper), which permhs 
detection of an audiorized usage. The odier feature on die screen. The Bump Size specification, 
pertains to die smallest unit of data, in die horizontal and vertical directions, diat die watermark 
algoritiuns operate on. The ectended information embedded in die watermark can be detected in 
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a manner analogous to that described with respect to Fig. 48, except that the addidonai 
infoimation is shown as illustrated in F^. 56. 

Aa alternative to a "plug in'' embodiment is a stand alone version, which is 
exemplified in the tuesxt sequence of figuxes. As discussed above, for any application that involves 

5 a watecmaric, preferably tbete is an autodeiect capability, and Fig. 57 illustrates a screen from a 
stand alone image reader widi a clip board triggered by merely putting an image on tbe clipboanL 
In die Clipboard Viewer window of Fig. 57» a watennaxk detected on the mage can be used to 
signal the user by one of several methods that can be used individually or in a combination. In 
one mediod, a tide bar icon or some other bar or icon alternates between "noimal" and 

10 "watermark detected** states periodically (ibr example, at one second intervals). Another ^roach 
is to use other state changes, like having the title bar or a task bar alternate between highlighted 
and nonnal stales periodically. StiU another method b to have die detection initiate a special 
dialog box display, as Ulnstrated in Fig. 53. The Read button m the special dialog box (trig^red 
by a clkk of a mouse, like oAer buttons) causes reading and display of die watermark 

15 information in the image. The Close button ends the special dialog. (So as not to interfere widx 
image processing, it can be convenient to show the special dialog box for a limited amount of 
time, and a small window therein can show the time remaining before the special dialog box 
disappears.) Buttons shown on the screen include one resembling a file folder to open a file; the 
next button is for copying a selected image born a first ^plication to the clipboard; the next 

20 button is for pasting an image &om the ciipboard to a second application; and the last is for help 
memis. 

Fig. 59 shows the image after pasting it from the cUjAoard, in the course of 
image processing. The Digimarc Reader - Image 1 window pops up on the screen to permit 
clicking die Web lookup button, which launches the user's web browser to result in a display of 
25 the creator's web page on the MarcCentre Web site, with die rest following as above. 

Xote ^pun that the foregomg approach is representative, and it can be modified 
for another media in which a watennark is used. For example, instead of an image, a sound can 
be used. 
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POTENTIAL U^^ QF THE TECHNOLOGY IN THE PROTECTION AhfD CQhTTROL QF 
jjIQFTWARE PROGRAMS 



Tbe illicit use, copying, and reselling of software programs represents a iiuge loss of 
5 revenues to the software industry at large. Tie prior an methods for attempting to mitiga^ this 
problem are veiy broad and will not be discussed here. What will be discussed is how the 
principles of ftis technology might be brought to bear on this huge problem. It is entirely unclear 
whether the tools provided by this technology will have any economic advantage (all things 
considecni) over the existii^ countermeasures both in place and contempiated. 



10 The state of technology over the last decade or more has made it a general necessity to 

deliver a &11 and complete copy of a software program m order for that program to function on a 
user's computer. In effect, $X were invested in creating a software program where X is large, 
and the entire fiuits of that development must be delivered in its entirety to a user in order for 
that leer to gam vahie ftom the software product Fortonately this is generally compiled codc^ 

15 but the point is that this is a shal^ distribution situation looked at in the abstract The most 
mundane (and harmless in die minds of most perpetrators) illicit copying and use of the program 
can be perftmned rather easily. 



This disclosure ofiara, at first, an abstract approach which may or may not prove to be 
economical m the broadest sense (where the recovered revenue to cost ratio would exceed that of 

20 most competing mediods, for cxamplcX Hie approach aqjands upon the m^oda and approaches 
already laid out in the sectiai on plastic credit and debit cards. The abstract concept begms by 
positing a "lafge set of unique patterns,'* uniqi» among themselves^ unique to a given product, 
and unique to a given purchaser of that proitact This set of patterns effectively contains 
thousands and even milUons of absotately unique "secret keys" to use the cryptology vernacular. 

25 Importantly and istinctly, these keys are non-deterministic, that is, riicy do not arise finom 

singular sub-lOOO or sub-2000 bit keys such as with the RS A key based systems. This large set 
of patterns is measured m kilobytes and Megabytes, and as mentioned, is oon^letenninistic in 
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nature. FuithOTdore; sdll at the most abstract level, these patterns are fully capable of being 
encrypted via standard tedmiqu&s and analyzed within the encrypted domain, where die analysis 
is made on only a small pomon of the large set of patterns* and diat even in die worst case 
scenario where a would-be pirate is monitoring the step*by-5tep microcode iostmctions of a 
5 micraprocessor, this gathered infionnation would provide no useful infbnnatioa to the would«be 
pirate. This latter point is an important one when it comes to "implementatioii security" as 
opposed CO "iimate security" as will be briefly discussed below. 

So what could be the di^rential properties of this type of key based system as opposed 
to, for example, the RSA cryptology mediods which are already well respected, relatively simple, 

10 etc. etc? As mentioned eariier, this discussion is not going to attetnpt a commercial stde<by-side 
analysis. Instead, we'll just focus on the differing propemes. The main distinguishing features 
M out in die impiementation itabn (die implemetitation security). One example is that in single 
low-bit-oumber private key systems, the mere local use and re-use of a single private key is an 
inherently weak link in an encrypted transmission systeuL ("Eooyptod transmission systems'* are 

15 discussed here m the sense that securing the paid-for use of software programs wUl in this 
discussion require de facto encrypted communication between a user of the software and the 
"bank" which allows the user to use the progiam; it is encryption in die service of electronic 
financial transacnons looked at in another light.] Would-be hackers wishing to defeat so-called 
secure systems never attack the fundamental hard-wired securi^ (the innate security) of the 

20 i^istine usage of the methods, they attack the implementation of those methods, centered around 
human nature and human oversights. It is hetr, still in the abstract, that the creaiion of a much 
larger key base, which is itself tton-detenntnisdc in nanire, and which is more geared toward 
effectively throw-away keys, begins to "idiot proof the more historically vulnerable 
impl«nentation of a given secure system. Tbe huge set of is not even comprehensible to Ae 

25 avOTge holcka- of those keys, and dieir use of dsose keys (i.e., the "implementation" of those 
keys) can randomly select keys, easily duow them out after a time» and can use diem in a way 
that no "eavesdroj^jer" will gain any useful in&nnadon in the eavesdropping, especially when 
well widiin a miUiomh of the amount of time that an eavesdropper could "decipher^ a key, its 
usefldness in the system would be long past. 
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Turning the abstract to the semi-concrete, one possible new approach to securely 
delivering a software product to ONLY the bonafide purchasers of that product is the following. 
In a mass economic sense, this new method is entirely founded upon a modest rate realtime 
digital connectivity (often, but not necessarily standard enciypted) between a user's computer 
5 ttetwo± and the selling company's network. At first glance this smells like trouble to any good 
maikcring person^ and indeed, diis may throw the baby out with the baAwarcr if by trying to 
recover lost revenues, you tose more legitimate revenue along the way (all part of the bottom line 
analysis). This new mediod dictates that a company selling a piece of software supplies to 
anyone who is willing to take it about 99.8% of its fimctional software for local storage on a 

10 user's network (for speed and minimizing transmission needs). This "free core program" is 
entirely unfuncrional and designed so that even the craftiest hackers can't make use of it or 
"decompile it" in some sense. Legitimate activation and use of this program is perftjtmed purely 
on a instruction-cycle-count basis and purely in a simple very low overhead coramunicaiioos basis 
between the user's network and the company*s network. A customer who wishes to use the 

15 product seids payment to the company via any of the dozens of good ways to do so. The 
customer is seat, via common shipment methods, or via commonly securwi encrypted data 
cfaanneis, their "huge set of onique secret keys." If we were to look at this large set as if it were 
an image, it would look just like the snowy images discussed over and over again in other parts 
of this disclosure. (Here; the "signature" is the im^e, rather than being imperceptibly placed 

20 onto another image). The special nature of this large set is that it is what we m«ht call 

"ridiculously unique" and contains a large number of secret keys. (The '^ridiculous" comes from 
toe simple math on Ae number of combinations that are posMble widi, say 1 Megabyte of random 
bit values, equaling exactly the number that "^all ones" would give, thus 1 Megabyte being 
approximately 10 raised to the -2,400,000 power, plenty of room for many people having many 

25 thiowaway secret k^). It is important to re-emphasize that the purchased entity is literally: 

productive use of the tool. The marketing of this would need to be very liberal in its aUocment of 
this productivity, since per-use payment schemes notoriously turn off users and can lower overall 
revenues significantly. 

This large set of secret keys is itself encrypted using standard encryption techniques. The 
30 basis for relatively higficr "implemenracion security" can now begin to manifest itself. Assume 
diat the us» now wishes to use die software product They fire up the free core, and the free 
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core program finds that die user has installed dieir large set of unique encrypted keys. The core 
program calls the company network and does the usual handshakmg. The company network, 
knowing the large set of keys belonging to that bonafide user, sends out a query on some simple 
set of patterns, almost exactly Ac same way as described in the section on die debit and credit 

S cards. The query is such a small set of the whole, that the inner workings of the core program do 
not even need to decrypt the whole set of keys, only certain parts of tb& key^ thus no decrypted 
version of die keys ever exist, even within die machine cycles on die local computer itself. As 
can be seen, diis docs not require the •signatures witiiin a picture'* methods of the main 
disctosure, instead, the many unique keys ARE die picwrc. The cone program interrogates die 

10 keys by performing certain dot products, dicn sends the dot products back to die company's 
network for voificatkjn. See Fig. 25 and die accompanying discussion for typical details on a 
verification transaction. Generally encrypted verification is sent, and the «« program now 
"enabtes" itself id pex&im a cenain amount of instntctions, for example, allowing 100,000 
characters being typcd mto a word processing program (before anodier unique key needs to be 

15 transmitted to enable anodier 100,000). la diis example, a purchaser may have bought the 
number of insttuctions which arc typically used within a one year period by a single user of die 
word processor pregram. Tlie purchaser of this product now has no incentive to "copy" the 
pro gr am and give it to their fiiends and relatives. 

Ail of die above is well and fine except &r two simple problems* The fint problem can 
20 be called "die ctoning problem" and the second *dic big brodicr problem.* The solutions to these 
two problems arc intimately linked. Tlie latter problem will ultimately become a purely sodai 
problem, with certam ledmical sohitioos as mere tools not ends. 

The dotting problem is die following. It gencniUy applies to a more sophisticated piraie 
of software rather dian the currendy commcai "friend gives dieir distribution CD to a friend" kind 

75 of piracy. Ciafty-hacker "A" knows diat if she p er for ms a system-state clone of the "enabled" 
program m its endxtty and mstails diis clcme on anodier machine, then this second machine 
effectively doubles die value received kr the same money. Keeping diis clone m digital storage, 
hacker "A" only needs to recall it and reinstall die clcme after the first period is nm out. dius 
indefinitely using die program for a single payment, or she can give the clone to dieir hacker 

30 frittid "B" for a ax-pack of beer. One good solutkm to diis problem requires, again, a radier well 
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developed and low cost real time digital connectivity between user site and company enabling 
networic This ubiquitous connecdvity generally does not exist today but is &st growing thmugh 
the Internet and the basic growth In digital bandwiddi. Part and parcel of the '*enabiing" ts a 
negligible conununicatkms cost random auditing ftuiction wherein the fimcticming program 
5 routxoeiy and irregularly perfcrms handshakes and veri&ations wtdi the company network. It 
does so, on average, during a cycle which includes a rather small amount of productivity cycles 
of the program. The cesulting average productivity cycle is b general much less than the raw 
total cost of the doming process of the overall enabled program. Thus, even if an enabled 
program is cloned, the usefiihiess of that mstantaneous clone is hi^ly limited, and it would be 

10 much more cost efibctive to pay the asking prke of die selling company dian to repeat die 
cloning process on sodi short time p^ods. Hackers couid brecdc this system for fim, but 
certainly not for profit The fitp side to this anaagement is that if a program '^calls up** the 
company's network Gar a random audit, die alkitted productivity count for that user on that 
program is accounted for, and that in cases where bonafide payment has not been received, die 

IS company network simply withhoMs its verificadon and die program no longer fimctions. We're 
back to vrfwrc users haye no mceative to "give diis away" to fticnds unless it is an explicit gift 
(which probably is quite appropriate if thqf have mdeed paid for it *'do anydimg you like widi it. 
you paid for rt^. 

The secfflid problem of "big brother^ and die hmiitively mysterious "enabling" 
20 communications between a user's network and a company's network would as mentioned be a 
social and perceptual problem diat should have all manner of potential real and imagined 
solutions. Even with the best and objectively unbeatable anti-big-brother solutions, there will still 
be a hard-core conqiiracy dieory crowd claimmg it just ain't so- With this in mind, one potential 
solution is to set up a single program registry which is largely a public or non-profit institution to 
25 handling and ooortQnadng die realtime verification networks. Such an entity would dien have 
company clients as well as user clients. Aa organizatkm such as the Software Publishers 
Association^ for example, may choose to lead such an eiforL 

Conchiding diis section, it should be re-cmpbasizcd titat tiw mediods here outi'med require 
a highly connected distributed system, in odicr words, a more ubiquitous and mcxpraisive Interact 
30 than exists m mid 1995. Simple trend extrapolation would argue diat this is not too Jar off from 



SOBSirrUTE sheet (rule 2B) 



wo 97/43736 



PCT/USJ7AM051 - 



- Wi 

1995. The growth tate in law digital omimiiaiatttons bandwidth also aigues that the above 
system might be more pf^cal^ sooner, than it might first appear. (The prospect of interactive 
TV brings with it the promise of a fast network linidng millions of sites aiound the world.) 



5 USE OF CURRHNT CRYPTOLQGY METHODS EN CONJUNCTICH^ WITH THIS 

TRrHNQLQGY 

It should be briefly noted diat cettain implementations of Ibe principles of this technology 
probably can make good use of current cryptographic tedmoiogies. One case in point might be a 
system wherdyy graphic artists and digitai photographers perfonn reahime registration of their 

10 photographs with die copyright office. It mi^ be advantageous to send the master code signals, 
or some representative poiticm thereof, directty to a third party registry. In this case, a 
photographer would want to know that their codes were being transtnitted securely and not stolen 
abng the way. In this case, certain common cryptographic transmission might be employed. 
Al50» photographers or mnsicians, or any users of diis technology, may want to have reliable time 

15 stamping services which are becoming more common. Such a service could be advant^;eousiy 
used in conjunction widi die prmciples of dus technology. 



r^KTATLS ON THE LEGITIMATE AND ILLEGniMATE DETECTT ON AND REMOVAL OF, 
TNVTSTRLE SIGNATURES 

20 In general, if a given entity can recognize die signatures hidden whhm a given set of 

mpirical data, that sxoe entity can take steps to remove those signatures. In practice, the degree 
of difiiculty between the former condition and the latter condition can be made quite large, 
f«tunately. Oo one extreme, one could posit a software program which is generally vay lUfficult 
to "decompile" and which does recognition functions on empirical data. This same bit of 

25 software couU not generally be used to "strip* die signanires (without going to extreme lengths). 
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On Che other band, if a hacker goes to the trouble of discovering and understanding the "public 
codes" used within some system of dam intexcfaange» and that hacker knows how to recogniK the 
signanires* it wouki not be a large for that hacker to read in a given sec of signed data and 
create a data sec with the signatures ei&ctivcly removed. In this latter example, interestingly 
5 enough, there would often be telltale smtistics that signatures had been removed, statistics which 
will not be discussed here. 



These and other such attempts to remove the signatures we can refer to as illicit attempts. 
Current and past evolution of die copyright laws have gcaerally targeted such activity as coming 
under criminal activity and have usually placed such language, along with penalties and 

10 enforcement language, into the standmg laws* Presumably any and all practitioners of this 
signaoire technology will go to lengdis to make sure that the same kmd of a) creation, b) 
distribution, and c> use of these kinds of illicit removal of copyright protection mechanisms are 
criminal offenses subjea to enforcement and penalty. On the odicr hand, it ban object of this 
technology to point out diat dnou^ the recogniti« steps outlined in this disclosure, software 

15 programs can be made such that the recognition of signatures can simply lead to their removal 1^ 
mverting the known signanircs by the amount equal to their found signal energy in the 
lecognition process (i.e., remove the size of the given code signal by exact amount found). By 
pointing this out in this disclosure, it is clear that such software or hardware which performs this 
signature removal operation will not only (presumably) be crimmal, but it will also be liable to 

20 infringement to die eJCtent dat it is not property Ucensed by the hoWers of the (j»esumably) 
patented technology. 



The case of legitimate and normal recognition of die signatures is strai^orwanL In one 
example, the public signatures could dehTjerately be made marginaUy visible (i.e. their intensi^ 
would be deliberately high), and m diis way a form of sending out "^of comps" can be 
23 accomplished. "Comps" and "proofe" have been used in the photographic industry for quite some 
timej where.a degradsd image is purposely sent out to prospective cuswmers so that tiicy might 
evahiate it but not be able tD use it in a commerdally meaning&l way. In the case of this 
technology, increasing the irteisity of the public codes can serve as a way to '^degrade" the 
commercial vahie mtenrionally, then through hardware or software activated by paying a purchase 
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price for the ma£eria]^ die public signatures can be removed (and possibly replaced by a aew 
invisible tracking code or sigoalucc, public and/ar private. 



MONTTORING STATIONS AND MONITORING SERVICES 

5 Ubiquitous and cost effective recognition of signatures is a central issue to the broadest 

pjoliferation of the principles of this technology. Several sections of this disclosure deal with this 
topic in various ways. This section focuses on the idea that entittes such as monitoring nodes, 
monitoring stations, and monitoring agencies can be created as part of a systematic enforcement 
of the principles of the technology. In order for such entities to operate, they require knowledge 

10 of dlie master codes» and they may require access to empirical data in hs raw (unencrypted and 
untransformed) form. (Having access to originaJ unsigned anpirical data helps in finer analyses 
but is not necessary.) 

Thice basic forms of monitoring statioos Ml out directly fiom the admittedly arbitrarily 
defined classes of master codes: a private monitoring station^ a semi-public, and a public. The 

15 distinctions are simply based on the knowledge of the master codes. An example of die fiilty 
private monitoring station might be a iai^ge photographic stock house which decides to place 
certain basic pAtems into its distributed material which it knows diat a tnify ciafly pirate could 
decipher and remove, but it thinks tiiis likelihood is ridiculously small on an economic scale. 
This stock house hizes a pat-time person to come in and randomly check high vahie ads and 

20 other photography in the public domain to search for these relatively easy to find base pattems» as 
well as cfaeddng phocog)ra[dis that stock house staff memben have **5potted" and think it might be 
infiingement material. The part time person o^mks through a large stack of these potential 
mfringemcn: cases in a few hours, and where the base patterns are found, now a more thorough 
analysis takes place to locate the original image and go duough the foil process of unique 

25 identification as outlined m this disclosure. Two core economic values accrue to the stock house 
in doing this* values which by definition will outweigh the costs of the monitoring service and the 
cost of the signing process iiseif. The first value is in letting their customers and die worid know 
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that they are signing their material and that the monitoring sm^ice is in place; backed up by 
whatever sxatisiics on the ability to catdx infringers. This is the deterrent value, which probably 
will be the largest value eventually. A genciai pre-rcquisite to this first value is the aoual 
recovered royalties derived from the monitoring eifort and its building of a track record for being 
5 formidable (oihancing the first value). 

The semi-public monifonng stations and the public monitoring stations largely follow the 
same pattern, although in these systems it is possible to actually set up diird party services which 
are given knowledge of the master codes by clients, and the services merely fish through 
thousands and millions of "creative property'* hunting for the codes and repoitit^ the results to 
10 the clients. ASCAP and BMl have "lower tech" approaches to diis basic service. 

A large coordinated monitorii^ service using the principles of this tedmology would 
classify its creative property supplier clients into two basic categories, those that provide master 
codes themselves and wish the codes to remain secure and unpublished, and those diat use 
generally public domain master codes (and hybrids of the two, of course}. The monitoring 

15 service would perform daily samplings (checks) of pnbliciy available imagery, video, audio, etc, 
doing high level pattern checks with a bank of supercomputers. Magazine ads and images would 
be scanned m fbr analysis, video grabbed off of commercial channels would be digitized, audio 
would be sampled, public Internet sites randomly downloaded, etc. These basic data streams 
would then be fed into an cver-cfauraing monitoring program which randomly looks for pattern 

20 matches between its large bank of public and private codes, and the dm material it is checking. 
A small sub-set, which itself will probably be a large set, will be flagged as potential match 
candidates, and these will be fed into a more r^ed checking system whidi begins to attempt to 
identify which exact signanires may be present and to perfOTn a more fine analysis on the given 
flagged material Piesumably a small set would then fell out as fl^ed match material, owners 

25 of that material would be positively identified and a monitoring report would be sent to the client 
so that they can verify that it was a legitimate sale of their material. The same two values of the 
private monitoring service outlined above apply in this case as well The monitoring service 
could also serve as a formal bully in cases of a found and proven infringement, sending out 
l«ters to infringing parties witnessing the found infringement and seeking inflated royalties so 

30 that the infringing party might avoid die more costly aitemacive of going to court 
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METHOD FOR EMBEDDING SUBLIMINAL REGISTRATION PATTERNS INTO IMAGES 
AND OTHER SIGNALS 

The vcxy nocton of reading embedded signatures invaives the concept of legistmion. Hie 
underiymg master noise signal must be known, and its relative position needs to be ascertained 

5 (registered) in order to initiate die reading process itself (e.g. the reading of die 1 's and O's of die 
N*bit identtficadon word). When one has access to the original or a diumbnai] of the unsigned 
signal^ tfiis registration process is quite srra^tforward. When one doesn't have access xo this 
signal, which is often the case in universal code applications of this technology, then different 
methods must be employed to accomplish this registration step. The example of pre-maiked 

10 photographic fihn and paper, where by definition there will never be an •^unsigned" original, is a 
pei&ct case in point of die latter. 

Maziy earlier sections have variously discussed this osue and presented certain solutions. 
Notably, the section on "simple" universal codes discusses one embodiment of a solution where a 
given master code signal is known a priori, but hs precise location (and indeed, it existence or 

15 non-existeiK») is not known. That particttlar section went on to give a specific example of how a 
very low level designed signal can be embedded widiin a much larger signal, wherein this 
designed signal is standardized so that detection equipment or reading processes can search for 
this standardized signal even diough its exact locarion is unknown. The brief section on 2D 
universal codes went on to point out that this basic concept could be extended into 2 dimensioas, 

20 or, eSecavely, into imagery and motion pictures. Also, the section on symmetric patterns and 
noise patterns outlined yet another approach to die two dimensional case, whcxein the nuances 
associated with two dimensional scale and rotation were more expiichly addressed, Thercin»_lhc 
idea was not merely to determme the proper orientation and scale of underiying noise patterns, 
but 10 have information transmitted as wclL eg-, the N rings for the N^ii identification woid. 

25 This section now attempts to isolare the sub-problem of registering embedded patterns fijr 

registration's sake- Once embedded patterns are registered, we can fcen look again at how 4is 
registration can serve broader needs. This section presenis yet anodier technique for embedding 
patterns, a technique which can be referred to as "subliminal digital graticules'. "Graticules'' - 
odier words such as fiducials or reticles or hash marks could just as well be used -conveys the 
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idea- of calibration marks used for the puiposes of tecating and/or measuring something. In this 
caset they are employed as low level patterns which serve as a kind of gridding function. That 
gridding function itself can be a carrier of 1 bit of information, as in the univetsal I second of 
noise (its absence or presence, copy me, don't copy me), or it can simply find the orientation and 
5 scale of other information, such as embedded signatures, or it can simply orient an image or audio 
object itsei£ 

Figs. 29 and 30 visually sammarize two related methods which ilhistrate applicant's 
subliminal digital graticules. As will be discussed, the method of Fig. 29 may have slight 
practical advantages over the method outlined in Fig. 30, but bodi methods effectively decompose 
10 the problem of finding the orientation of an image into a series of steps which converge on a 
solution. The prd)lcm as a whole can be singly stated as the following: given an arfjitrary 
image wherein a sublirainai digital graticule may have been stamped, then find the scale, rotation, 
and origm (of&et) of the subliminal digital graticule* 

The beginning point for sublunmal graticules is in defining what they are. Simply put, 
15 they are visual patterns which are directly added into other images, or as the case may be, 
exposed onto jAotographic fifan or paper. The classic double exposure is not a bad analogy, 
thou^ in digital Imaging this specific concept becomes rather stretched. These patterns will 
generally be at a very low brightness level or exposure IcveL such that when they arc combined 
with "nonnal" images and exposures, they will effectively be invisible (subliminal) and just as the 
20 case with embedded signatures, they will by definition not interfere with the broad value of the 
images to which they are added. 

Figs. 29 and 30 define two classes of subliminal graticules, each as represented in the 
spatial fiecpjcncy domain, also known as the UV plane, 1000. Common two dimensional fourier 
transforai algorithms can iranrfbrm any given image into its UV plane ccmjugatc. To be precise. 
25 the depictions b Figs. 29 and 30 are the magnitudes of rhc spatial frequencies, whereas it is 
difBcolt to depict the phase and magnitude which exists at every point 
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Fig. 29 shows the example of six spots in each quadrant along the 45 degree lines, 1002. 
These are exaggenued in this figure, in that these spots would be difficult to discern by visual 
inspection of the UV plane image. A rough depiction of a "typical" power spectrum of an 
arbitrary image as aiso shown, [004. This power spectrum is generally as unique as images are 

5 unique. The subliminal graticules are essentially these ^ots. In diis example, d^re are six 
spatial frequencies combined along each of the two 45 degree axes. The magnimdcs of the six 
frequencies can be the same or different (we'll couch upon this refmement later). Generally 
speaidng, the phases of each are different from the others, including the phases of one 45 degree 
axis relative to die other. Fig. 3 1 depicts this graphically. Hie phases in this example are simply 

10 randomly placed between PI and -PI, lOOS and 1010. Only two axes ore represented in Fig. 31 - 
as opposed to the four separate quadrants, since the phase of the miirored quadrants are simply 
?y2 out of phase with their mirrored counterparts. If we tumed up the intensity on this 
sublaninal graticule, and we tiassfeimed the result mto the inuge domain, then we would see a 
weave-like cross-hatching pattern as related m the caption of Fig. 29. As stated^ this weave-like 

15 pattern would be at a very low intensity when added to a given ims^e. The exact frequencies and 
phases of the spectxal components uciiiaed would be stored and standardized. These will become 
the "spectral signatures" that registration equipment and readutg processes will seek to measure. 

Briefly, Fig. 30 has a variation on this same general dieme. Fig. 30 lays out a diSeient 
class of graticules in that the spectral signature is a simple series of concentric rings rather than 

20 spots along the 45 deg^ axes. Fig. 32 then depicts a quasi-random phase profile as a function 
along a half-«ircle (the other half of the circle then being FI/2 out of phase with the first half). 
These are simple examples and dierc are a wide variety of variations possible in designmg the 
phase praBks and the radii of the concentric rings. Hie transfomi of this type of subliminal 
graticule is less of a "pattern* as with tiiie weave-like graricule of Fig. 29, where it has more of a 

25 random appearance like a snowy image* 

The -idea behind both types of graticules is the ibilowing: embed a unique pattern into an 
image which virtually always will be quite distinct from the imagery into which it wOl be added, 
but which has certain propeittes which &ciiitate &st location of the pattern, as well as accuracy 
properties such that when the pattern is generally located, its precise locarion and orientation can 
30 be found to some high level of precision. A corollaiy to the above b to design the pattern such 
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that the panem on average minimally interferes with the typical imagery into which ii will be 
added, and has maximum energy relative to the visibility of the pattem. 

Moving on to the gross summary of how the whole process works, the graticule type of 
Fig. 29 &cilttates an image processing search which begins by first locating the rotation axes of 
S the subliminal graticule, then locating the scale of the graticule^ dien determining the origin or 
of&et The hst step here idendfies which axes is which of the two 45 degree axes by 
determining phase. Thus even if the image is largely upside dowi^ an accurate determinadon can 
be made. The Gtsc step and the second step can both be accomplished using ody the power 
spectrum data, as opposed to the phase and magnitude. The phase and magnitucb signals can 

10 then be used to "fine tune"* the search for the correct rotation angle and scale. The graticule of 
Fig« 30 switches the first two steps above, where the scale is found first, then the rotation, 
followed by precise determmation of the origm. Those skilled in the art will recognixe that 
detennining these outstanding parameters, along two axes, are sufficient to fully register an 
image. The ''engineering opthnizatioa challenge** is to maximize the uniqueness and brightness of 

15 the patterns relative to then* visibility, while minimizing the computational overi:tfsd in rc^dimg 
some specified level of accuracy and precision m registration. In the case of exposing 
photographic fihn and paper, clearly an additional engineering challenge is the outlining of 
economic steps to gee die patterns exposed omo the film and paper in the first place, a challenge 
which has been addressed in previous sections. 

20 The problem and solution as above defined is what was meant by registradon for 

regisiradon's sake. It should be noted that there was no mention made of making some kind of 
value judgement on wh^er or not a graticule is indeed being found or not Clearly, the above 
steps could be applied to unages which do not in &ct have gzadcules inside them, the 
measurements then simply chasing noise. Sympathy needs to be extended to the engineer who b 

25 assigned die task of setdng ''detection thresholds" for these types of patterns, or any others, amidst 
the incredibly broad range of tmagety and eovhonmental condidons in which die patcems must be 
sought sid verified. [Ironically, this is where the pure universal one second of noise stood in a 
previous section, and that was why it was appropriate to go beyond merely detecting or not 
detecting this singular signal ix. adding additional infbrmadon planes]. Hotin is where some 

30 real beauty shows up: tn the combination of the subluninal graticules with the now-registered 
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embedded signatures described in other parts of this disclosure. Specifically, once a "candidate 
registranon'* found - paying due homage co the idea chat one may be chasing noise - then the 
next logical step is to pcrfoxra a reading process for, e.g., a 64 bit universai code signature. As 
furdier example, we can imagine that 44 bits of the 64 bit identificdtion word are assigned as an 

5 index of registered usen - serial numbers if you will. The remainmg 20 bits ane reserved as a 
hash code - as is well known in encryption arts - on the 44 bit identification code thus found 
Thus, in one swoop, the 20 bits serve as the ''yes, I have a registsed image" or "no, I don't" 
answer More importantty, perhaps, this allows for a system which can allow for maximum 
flexibilhy in precisely defining the levels of '^felsc positives*' in any given automated 

]0 id^tification system. Thr^oid based detection will always be at the mercy of a plethora of 
conditions and situadons* uidmateiy resting on arbitrary decisions. Give me N coin flips any day. 

Back on point, these graticule patterns must fii^ be added to an image, or exposed onto a 
piece of film. An exonplary program reads in an arbitrarily sized digital image and adds a 
specified graticule to the digital image to produce an output image. In the case of fihn, the 
15 graticule pattcra would be physically exposed onto the fihn either before, during, or after 

exposure of die primary image. All of these methods have wide variations in how they might be 
accomplished. 

The seardiing and registering of subliminal graticules is the more interesting and involved 
process. This section will Srst describe the elements of this process, culminating in the 
20 generalized fiow chart of Fsg. 37. 

Fig. 33 depicts the first major "search" step in the registration process for graticules of die 
type in Fig. 29. A suspect image (or a scan of a suspect photograph) is first transfonned in its 
fourier representation using well known 2D FFT routines. The input image may look like the one 
in Fig. 36, upper left image. Fig. 33 conceptually represents the case where the image and hence 
25 the graticules have not been rotated, though the foUowing process fiilly copes with rotation issues. 
After the suspect image has been transformed, the power spectrum of the transform is then 
calculated, being simply the square root of the addition of the two squared moduli it is also a 
good idea to perfinm a mild low pass filter operation, such as a 3x3 blur filter, on the resulting 
DOwer soectrum data, so that later search fstenst dftn'f nettA inrrpHihIv fin^ enM^rf c»p»ic ttm^w th^ 
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caixiidate lotatioa angles fitun 0 throQ^ 90 d^rees (or 0 to PI/2 in radian) are stepped through. 
Along any given angle, two resultant vectors are calculated, the first is the simple addition of 
power spectrum values a a given cadhiS along the four lines emanatii^ from the origin in each 
quadrant The second vector is the moving average of the first vector. Then, a normalized power 
S profile is calculated as depicted in both 1022 and 1024, the difference being that ox» plot is along 
an angle which does not align widi the subliminal gradcules, and the other plot does align. The 
normalization stipulates drat the fim vector is die numerator and the second vector is the 
denominator in the resultant vector. As can be seen In Fig. 33, 1022 and 1024, a series of peaks 
(which should be '*six" instead of "five" as is drawn) develops when the angle aligns along its 

10 proper direction. Detection of these peaks can be efSected by setting some threshold on the 

normalized values, and integrating dietr total alcmg the whole radial line. A plot, 1026, from 0 to 
90 degrees is depicted m the bottom of Fig. 33, showing that the angle 4S degrees contains die 
most energy. In practice, this signal is often much lower than diat depicted in diis booom figure, 
and mstead of picking the highest value as the "found rotation angle," one can simply find the top 

15 few candidate angles and submit these candidates to the next stages in the process of determming 
the xegistration. It can be apprecialied by those practiced in die art that the foregoing was simply 
a known signal detection scheme, and that there are dozens of such schemes that can ultimately 
be created or borrowed. The simple requirement of the first stage process is to whittle down the 
^didate rotation angles to just a few, wherein more refined searches can then take over. 

20 Fig. 34 essentially outlines the same type of gross searching in the power spectral domain. 

Here instead we first search for the gross scale of the concentric rings, stepping from a small 
scale dirou^ a targe ^ak, rather than the rotation angie. The graph depicted in 1032 is the same 
normalized vector as in 1022 and 1024, but now the vector values are plotted as a function of 
angle around a semt-<ircle. The moving average denominator still needs to be calculated in the 

25 radial directioj, radier than die tangential direction. As can be seen, a similar "peaking" in the 
normalized signal occxn:s when the scanned circle coincides with a graticule circle, giving rise to 
the plot 1040. The scale can dien be found on die bottom plot by matching die known 
characteristics of the concentric rings (l.e. their radii) with the profile in 1040. 

Fig. 35 depicts the second primary step in regislerir^ sublnninal graticules of die type m 
30 Fig. 29. Once we have found a few rotanon candidates from the methods of Fig. 33, we dien 
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take the plots of the candidate angles of the type of 1022 and 1024 and peribnn what the invemor 
rders to as a "scaled li»iner matched filtering operation on those vectors. The scaled kemei 
refers to the &ct that the kernel in this case is a known non-hannonic relationship of frequencies, 
represented as the lines widi x's at the top in 1042 and 1044. and that the scale of these 

5 frequencies is swept through some pre-decermined range, such as 25% to 4009^0 of some expected 
scale at 100%. The matched filter operation sunply adds the resultant muitipKed values of die 
scaled fiequencies and their plot counterparts. Those practiced in the ait will recognize die 
sunilaiiry of this opeiadon with the very well known matched filter operation. Tlie resulting plot 
of the matched filter opoadon will look something like 1045 at the bottom of Fig. 35. Each 

10 candidate angle from the first step will generate its own such piot, and at this point the highest 
value of all of die plots will become our candidate scale, and the ai^ie corre^nding to the 
highest vaiue will become our primary candidate rotation angle. Likewise for graticules of die 
type in F^. 30, a similar "scated-kemer matched filtering operation is performed on the plot 
1040 of Fig. 34. This generally provides for a single candidate scale foctor. Then, using die 

15 stored phase plots 1012, 1014 and 1016 of Fig. 32, a more traditionai matched filtering operation 
is applied between diese stored plots (as kemelsX and die measured phase profiles along the half- 
circles at die previously finmd scale. 

The hist step in roistering graticules of the type in Fig. 29 b to pcrfomi a garden variety 
matched filter between die known (either spectral or spatial) profile of the graiicule widi die 

20 suspect image. Smce both die rotation, scale and orientation arc now known from previous steps, 
this matched filtering iteration is strai^tforward. If the accuracies and precision of preceding 
steps have not exceeded design specifications in the process, &ai a simple micro-search can be 
performed m die small neighbcjrfiood about the two parameters scale and rotation, a matched filter 
operation performed* and the highest value found will determine a "fine omed" scale and rotation. 

25 In this way, die scale and rotation can be fijund to widim the degree set by the noise and cross- 
talk of die suspect onage itself. Likewise, once the scale and rotation of the graticuies of die type 
in Fig. 30 are found, then a straightforward matched filter operation can complete the registration 
process, and similar "fine tuning" can be applied 

Movmg on to a variant of the use of die graticules of die type in Fig. 29, Fig. 36 presents 
30 the possibility for finding the subliminal gradcules without die need for perfornimg a 
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cotnputatiocally expensive 2 dimensional FFT (fast fourier transform). \n situations where 
computatioaai overhead is a major issue, then (he search problem can be reduced to a series of 
one*dimensional steps. Fig, 36 bn>adly depicts how to do this. The figure at the top left is an 
arbitrary image in which the graticules of the type of Fig. 29 have been embedded. Starting at 
5 angle 0, and finishing with an angle just below 180 degrees, and stepping by, for example 5 
degrees, the grey values along die depicted columns can be simply added to create a resulting 
column-integrated scan« 1058. The figure in the top ri^t, 1052, depicts one of the many angles 
at which this will be performed This column^integrated scan then is txansfbrmed into hs fourier 
representation using the less computationally expensive 1 dimensional FFT. This is then turned 

10 into a magnitude or "power" plot (the two are different), and a similar normalized vector version 
created just like 1022 and 1024 in Fig. 33. The difference now is that as the angle approaches 
the proper angles of the graticules, slowly the tell-tale peaks begin to appear in the 1024- like 
pIots» hut they generally show up at higher fi^uencies than expected for a given scale, since we 
are generally slightly off on our rotation. It remains to find the angle which maximizes the peak 

15 signals, thus zooming in on the proper rotation. Once the proper rotation is found, then the 

scaled kernel matched filter process can be applied, followed by traditional matched filtering, ail 
as previously described Again, the sole idea of the "short-cut" of Fig. 36 is to greatly reduce the 
computational overhead in using the graticules of the type in Fig. 29. The mventor has not 
reduced this method of Fig. 36 to practice and currendy has no data on precisely how much 

20 computational savings, if any, will be realized. These efforts will be part of application specific 
development of the method. 

Fig. 37 simply summarizes, in order of major process steps, the methods revolving around 
the graticules of the type in Fig* 29. 

In another variant embodiment, the graticule energy is not concentrated around the 45 
25 degree angles in the spatial frequency domain. (Some compression algorithms, such as JPEG, 
tend to particulariy atcmuate nnage energy at this orientation.) Instead the energy is more widely 
spatially spread Fig. 29A shows one such distribution. The frequencies near the axes, and near 
the origin arc generally avoided, since this is where fte image energy is most likely concentrated. 
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Detection of this ^crgy in a suspect image again relies on techniques like that reviewed 
above. However, instead of firet identifying the axes, then the rotation, and then the scaJe, a 
more global pattern matching procedure is perfonncd in which all are detennined in a brure force 
effoTL Those skilled in the art will recognize that the Fourier-Meliin tiansform is well suited for 
S use in such pattern matching problems* 

The foregoing principtes Bad appiicatioii, for example; in pfaoio-dupltcation kiosks. Such 
devices typically uiclude a lens for imaging a customer-provided original (e.g. & photographic 
print or BIm) onto an opto-electronic detector, and a print-writing device for exposing and 
developing an emulsion substrate (again photographic paper or fibn) in accordance with the image 
10 dam gathered by die detector. The details of such devices are well known to those skilled in the 
art, and are not belabored here. 

In such systems, a memory st<H^ data from the detector, and a processor (e.g. a Pentium 
micioprocessor with associated support components) can be used to process the memory data to 
detwt the presence of copyright data steganographically encoded therein. If such data is detected, 
15 the print-writing is intemipted. 

To avoid defeat of the system by manual rotation of the original image off-axis, the 
processor desirably implements the above-described technique m efFect automatic registration of 
the original notwithstanding scale, rotation, and origin offset factors. If desired, a digital signal 
processing bood can be employed to oflBoad cmain of the FFT processing from the main (e.g. 
20 Pcnthan) processor. After a rotated/scaled image is registered, detection of any 

steganographically encoded copyright notice is straightforward and assures die machine will not 
be used in violation of a photogiaphePs copyright 

While the techniques disclosed above have make use of applicant's preftrred 
steganographic encoding m^ds, die principles thereof are ra(« widely applicable and can be 
25 used in many instances in which automatic registration of an image is to be effected. 
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USK OF EMBEDDED SIGNATURES IN VTOEO. WHEREIN A VTDEO DATA STREAM 
KFFFCTTVELY SERVES AS A HIGH-SPEED ONE WAY MODEM 

Through use of die univetsal coding system oudined in earlier sections, and through use 
of master snowy fiames whkh cliange in a simple fiisfaion fiame to fiame, a simple receiver can 

5 be designed such that h has pfe4uiow[edge of the dianges m the master snowy frames and can 
therefore read a changing N*btt message word fifame by frame (or I*fiame by I-&ame as the case 
may be in MPEG video). In this way, a motion picture sequence can be used as a high speed 
one-way infinmation channel^ much like a one*way modon. Consider, for example, a fiame of 
video data with N scan lines which is st^anographicaliy encoded to effect the transnissicm of an 

10 N-bit message. If there are 484 scan lines in a &ame (N), and frames change 30 times a second, 
an information channel with a capacity comparable to a 14.4 kilobaud modem is achieved. 

In actual practice, a data rare substantially in excess of N bits per frame can usually be 
achieved, yielding transmission rates nearer that of ISDN circuits. 



15 FRAUD PREVENTION IN WIRELESS COMMUNICATIONS 

In the cellular telephone industry, hundreds of millions of dollars of revenue is lost each 
year through theft of services. While some services axe lost due to physical theft of cellular 
telephones, the more pernicious threat is posed by cellular telephone hackers. 

Cdlular telqibone hackers employ various electronic devices to mimic the identification 
20 signals produced by an audiorized cellular telephone. (These signals are sometimes called 

autfaoriatfion signals, veriftcatioo numbers, sxgnann data, etc.) Often, the hacker learns of these 
signals by eavesdrqspmg on authorised cellular telephone subscribers and recording the data 
exchanged with the cell cite. By artfol use of this data* the backer can impersonate an autlurized 
subscriber and dupe the earner into completing pirate calls. 
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In the prior art, identification signals arc segregated from the voice signals. Most 
commonly, they are temporally separated, e.g. transmitted in a burst at die time of call 
origination. Voice data passes through the channel only after a verificatioa operation has cakea 
place on this identificactoa data. (Identification data is also coizijiH}nly included in data packets 
5 sent during the transmission.) Another approach is to spectrally separate the identification, e.g. in 
a spectral subband ootsidc that allocated to the voice data. 

Odier fraud-dererrent schemes have also been employed. One class of techniques 
mwiitors characteristics of a cellular telephone's RF signal to identify the originating phone* 
Another class of techniques uses handshaking protocols, wherein some of the data r^unicd by the 
10 cellular telephone is based on an algorithm (e.g. hashing) applied to random data sent thereto. 

Combinations of the foregoing approaches are also sometimes employed 

as. Patents 5.465487, 5,454,027, 5,420,910, 5,443,760, 5,335,278, 5,345,595, 
5,144,649, 5^04,902, 5,153^19 and 5,3881,212 detail various ceUular telephone systems, and 
fraud deterrence techniques used therein. The disclosures of these patents are incorporated by 
15 reference. 

As the sophistication of fraud deteirence systems increases* so does the sophistication of 
cellular telephone hadcers. Ultimately, hacken have the upper hand since diey recognize dm aU 
prior ait systems are vulnerable to the same weakness: the identification is based on some 
attribute of the cellular telephone txansmission outside the voice data. S'mce this attribute is 
20 segregated ftom the voice data, such systems will always be susceptible to pirates who 
electronically "patch" their voice into a composite etectnonic signal having the attribute(s) 
necessary to defeat die fraud deterrence system. 

To overcome this failing, p re fe rre d embodiments of this aspect of the present technology 
5teganogi?|McaIly encode the voice signal widi identification data, resulting in "In-band" 
25 signalling (in-band both temporally and spectrally). This approach allows the carrier to monitor 
the user's voice sigital and decode the tdeoti&ation data therefiom. 
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In one such fonn of the tschoology, some or all of the identification data used in the prior 
art (e.g. data transmitted at call origination) is nspeatcdly steganographically encoded in die user's 
voice signal as well. The carri^ can thus periodically or aperiodically check the identification 
data accompanying the voice dam with that sent at call origination to ensure they match. If they 
5 do not, the call is identified as being hacked and steps for remediation can be pistigated such as 
interrupting the call 



In another fbnn of die technology, a randomly selected one of several possibte messages 
is repeacedly steganographically encoded on the subscriber's voice. An index sent to the cellular 
carrier at call set-up identifies which message to expect If die message sieganographicaliy 
LO decoded by die cellular cairi«- from the subscriber's voice does not match that expected, the call 
is identified as fi^udulenL 



In a preferred form of this aspect of the technology, the steganographic encoding relies on 
a pseudo random data signal to transform die mess^e or identification data into a low level 
noise-Iike signal superimposed on the subscriber's digitized voice signal. This pseudo random 
15 data signal is known, or knowable, to both the subscriber's telephone (for encoding) and to the 
cellular carrier (for decoding). Many such embodiments rely on a deterministic pseudo random 
number generator seeded with a damm known to bodi the telephone and the carrier. In simple 
embodiments this seed can remain constant fixan one call to die next (e.g, a telephone ID 
number). In more complex embodimcats, a pscudo-onc^c pad system may be used, wherem a 
new seed is used for each session {i.e. telephone call). In a hybrid system, ibc telephone and 
eelhilar earner each have a i^rence noise key (e.g. 10,000 bits) fimn which the telephone selects 
a fidd of bits, such as 50 bits beginning at a randomly selected ofifeet, and each uses this excerpt 
as the seed to generate the pseudo random data for encoding- Daa sent fiom the telephone to the 
carrier (e.g- the offiet) during caU set-tip aUows the carrier to reconsmict the same pseudo 
25 random data for use m decoding. Yet further improvements can be derived by borrowing basic 
techniques bom the art of cryptographic communicaiions and applying them to the 
steganographically encoded signal detailed m this disclosure. 



20 
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Details of applicant's preferred techniques for steganogr^c encoding/decoding with a 
pseudo random data stream are more particulariy detailed in die previoiis portioQS of diis 
specificatioQ, but die present technology is not limited to use with such lecimiques. 

The reader is presumed to be fiamiiiar with celhilar communications technologies. 
5 Accordingly^ details known from prior an in this field aren*t belabored herein. 



Referring to Fig. 3S. an illustrative cellular system inclades a telephone 2010, a cell site 
Z012, and a central office 2014. 

Conce;xuaIly, die telephone may be viewed as including a microphone 2016, an A/D 
converter 201S» a data formatter 2020, a modulator 2022, an RF section 2024, an antenna 2026, a 
10 demodulator 2028, a data uoforxnansr 2030, a D/A converter 2032, and a speaker 2034. 



In operation, a subscriber's voice is picked up by the microphone 2016 and converted to 
digital fonn by the A/D converter 20 1 8. The data formatter 2020 puts die digitized voice into 
packet form, adding synchronization and control bits thereto. The moduialor 2022 converts this 
digital data stream into an analog signal ^ose phase and/or amplitude properties change in 
15 accordance with die data being modulated- The RF section 2024 commonly ffanslates diis time^ 
vaiying signal to one or more intermediate frequencies, and finally to a UHF transmission 
frequency. The RF secticm diereafter amplifies it and provides the resulting signal to the antenna 
2026 foe broadcast to the cell site 2012. 

The process woriu in reverse when receiving. A broadcast from the ceil cite is received 
20 dirougb die antenna 2026, RF section 2024 amplifies and translates the received signal to a 
diflFcrent frequency for demodulation. Demodulator 2028 processes die amplitude and/or phase 
variations of die signal provided by die RF section to produce a digital data stream corresponding 
thereto. The data unfonnattcr 2030 segregates the voice data from the associated 
synchronization/control and passes die voice data to the D/A converter for conversion into 
25 analog form. The output from die D/A converter drives the speaker 2034. dirougfa which die 
subscriber hears die other party's voice. 
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TIm celi site 2012 receives broadcasts from a plurality of teiephooes 2010. and relays the 
data received to the central office 2014. Likewise, the ceil site 2012 receives outgoing data from 
the centrai ofTice and broadcasts same to the telephones. 

The central oSice 2014 perfoims a variety of operations^ including call authentication, 
5 switching, and cell hand-off. 

(In some systems, the functional division between the cell site and the central station is 
diff^nt than that outlined above. Indeed, in some systems, all of this functionality is provided at 
a single she.) 

In an exanplary cmbodmieat of this aspect of the technology, each tdephone 2010 
10 additionally mcludes a steganographic encoder 2036. Likewise, each cell site 2012 includes a 
steganographic decoder 2038. The encoder operates to hide an auxiliary data signal among the 
signals representing the subscriber's voice. The decoder perfbnns the reciprocal function^ 
discerning the auxiliary data signJ from the encoded voice signal. The auxiliary signal serves to 
verify the legitimacy of die call. 

15 An exemplary steganographic encoder 2036 is shown m Fig. 39. 

The illustrated encoder 2036 operates on digitized voice data, auxiliary data, and pseudo- 
random noise (PRN) data. The digitized voice data is applied at a port 2040 and is provided, 
c,g., from A/D cOTverter 2018, TTie digitized voice may comprise 8-bit samples- The auxiliary 
data is applied at a port 2042 and comprises, in one form of the technology, a SU"^ of binary 

20 data uniquely identifying die telephone 20 1 0. (The auxiliary data may additionally include 
administrative dam of die sort conventionally cxjiiangcd witii a cell site at call set-up.) The 
pseudo-random noise data is applied at a port 2044 and can be, e.g., a signal diat randomly 
alternates between "-r and T values. (More and more cellular phones are incotporaring spread 
spectrum ca^xibie circuitry, and this pseudo-random noise signal and other aspects of this 

25 technology can often *^iggy-back- or share the circuitry which is alieady being applied in die 
basic operation of a cdluiar tmit). 
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FoT fixposttoty convenience, it is assumed that all three data signals applied to the encoder 
2036 are clocked at a common rate, although this is not necessary m practice. 

In operation, the auxiliary data and PRN data streams are applied to the two inputs of a 
logic circuit 2046. The output of circuit 2046 switches between -I and +1 in accordance with the 
5 following table: 



1 AUX 


PRN 


OUTPUT 


1 ^ 


-I 


1 


1 ° 


I 


•I 


I 


>l 


-1 


I 


I 


1 



(If the auxfliaiy data s^nai is conceptualized as switching between -I and 1, instead of 0 and 1, it 
will be seen that droiit 2046 operates as a one-bit multiplier.) 

15 lie output fiom gate 2046 is thus a bipolar data stream whose instantaneous value 

changes landomly in accordance with the corresponding values of the auxiliary data and the PKN 
data. It may be regarded as noise. However, it has die auxiliary dal?i encoded dicceia The 
auxiliary data can be extracted if the corresponding PRN data is known. 

The noise-like signal firom gate 2046 is applied to die iiqjut of a scaler circuit 2D4«. 
20 Scaler circuit scales (eg. moltipfies) dus input signal by a fector set by a gain control circuit 
2050. In the illustrated embodiment, diis fictor can range between 0 and 15. The output fiom 
scaler circuit 2048 can dxus be represented as a five-bit data word (four bits, plus a sign bit) 
which changes each clock cyde, in accordance with the auxiliary and PRN data, and die scale 
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Eactor. The output from the scaler circuit may be regarded as '*sca(ed noise data" (but again it is 
"noise'* fiom which the auxiliaiy data can be recovered, given the PRN data). 

The scaled noise daa is summed with the digitized voice data by a summer 205 1 to 
provide the encoded output signal (e.g. binarily added on a sample by sample basis). This output 
5 signal is a composite signal representing both the digitized voice data and tbs auxiliary data. 

The gain conmsl circuit 2050 controls the magnitude of the added scaled noise data so hs 
addition to the digitized voice data does not noticeably degrade the voice data when converted to 
analog form and heard by a subscriber. The gain control circuit can operate in a variety of ways. 

One is a logarithmic scaling function. Thus» for example, voice data samples having 
10 dedmai values of 0, i or 2 may be correspond to scale factors of unity, or even zero, whereas 
voice data samples having values in excess of 200 may correspond to scale &ctors of 15. 
Generally speaking, the scale factors and the voice data values correspond by a square root 
relation. That is, a four-fold increase in a value of the voice data corresponds to approximately a 
two-fold increase in a vahie of the scaling factor associated thcrewidi- Another scaling function 
15 would be linear as derived from the average power of the voice signal. 

(The parenthedcai reference to zero as a scaling factor alludes to cases, e.g., m which the 
digitized voice signal sample is essentially devoid of information content) 

More satis&ctoiy than basing the instantaneous scaling &ctor on a single voice data 
sample, is to base the sealmg factor on the dynamics of several samples. That is, a stream of 
20 digitized voice data which is changing rapidly can camouflage relatively more auxiliary data than 
a stream of digitized voice data which is changing slowly. Accordingly, the gain comroi circuit 
20S0 can be made responsive to the first, or preferably the second- or higher-order derivative of 
the voice data in setting the scaling factor. 

In still other embodiments, the gain control blodk 2050 and scaler 2048 can be omitted 

25 endrely. 
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(Those skilled in the ait will recognize the potential for "rail errors" in the foregoing 
systems. For example, if the digitized voice data consists of S-bit sampfes^ and the samples span 
the entire range jBnom 0 to 255 (decimal), then die addition or subtraction of scaled QOtse U)/from 
the input signal may produce output signals that cannot be represented by i bits (e.g. -2, or 257). 
5 A number of well-understood techniqties exist to rectify this situation, some of them proactive 
and some of them reactive. Among these known techniques are: specifying that the digitized 
voice data shall not have samples in the range of 0-4 or 241-255, thereby safely perraittmg 
combination with the scaled noise signal; and including provision for detecting and adaptively 
modifying digitized voice samples that would otherwise cause rail eriurs.) 

10 Returning to the telephone 2010, an encoder 2036 like that detailed above cs desirably 

interposed between the A/D converter 20 IS and the dam formatter 2020, thereby serving to 
steganographicaliy encode all voice transmissions with the auxiliary data. Moreover, the circuitry 
or software controlling operation of the telephone is arranged so that die auxiliary data is encoded 
repeatedly. That is, when all bits of the auxiliary data have been encoded, a pobtcr loops back 

15 and causes the auxiliary data to be applied to the encoder 2036 anew. (Tlie auxiliary data may be 
stored at a known address in RAM memory for ease of reference.) 

It will be recognized that the auxiliary data in the illustrated embodtment is transmitted at 
a race one-eighth djat of the voice data. That is, for every 8-bit sample of voice data, scaled 
noise data corresponding to a single bit of the auxiliary data is sent Thus, if vokc samples arc 

20 sent at a rate of 4800 sampfes/second, auxiliary data can be sent at a rate of 4800 bits/second* If 
the auxiliary data is comprised of 8-bit symbols, auxiliary data can be conveyed at a rate of 600 
symbols/second. If the auxiliary data consists of a string of even 60 symbols, each second of 
voice conveys the auxiliaiy data tea times« (Significantly higher auxiliary data rates can be 
achieved by resorting to mm eScient coding techniques, such as limited-symbol codes (e.g. 5- 

25 or 6-bit codes)» HufSnan codmg, etc,) This highly redundant transmission of the auxiliary data 
permits lower amplitude scaled noise data to be used while still providing sufiicient signal-to- 
noise headroom to assure reliable decoding - even in the rclativdy noisy environment associated 
with radio transmissions. 
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Tuming now to Fig. 40. each cell site 2012 has a sleganographic decoder 2038 by which 
it can analyze the composite data signal broadcast by the telephone 2010 to discOT and separate 
the auxiliaiy data and digitized voice data therefrom, (The decoder desirably works on 
unformatted data (i.e. data widi die packet overiiead, control and administrative bits removed; this 
5 is not showti for clarity of illustration). 



The decoding of an unknown embedded signal (i.e. the encoded auxiliary signal) from an 
unknown voice signal is best done by some form of statistical analysis of the composite data 
signal. The techniques therefor discussed above are equally applkable here. For example, the 
entropy-based approach can be utilized. In this case, the auxiliary dara repeats every 480 bits 

10 (instead of every 8). As above, die emropy-bascd decoding process treats every 480th sample of 
the composite signal in like fashion. In particular, the process b^ by adding to the 1st, 431st, 
861st, etc. samples of the composite data signal the PRN data with which tfiese samples were 
encoded. (Ttat is, a set of spaise PRN data is added: the original PRN set. with all but every 
430th datum zeroed out.) The localized entropy of the resulting signal around these points (Lc, 

15 the composite data signal whh every 480th sample modified) is then computed. 



The foregoing step is then repeated, this time subtracting the PRN data corresponding 
thereto from the 1st, 481st, 961si, etc composite data samples. 



One of these two operations will counteract (e.g. undo) the encoding process and reduce 
the resulting signal's entropy; the other will aggravate it If addmg Ae sparse PRN data to the 
20 composite data reduces its entropy, then this data must earlier have been subtracted ftom the 
original voice signal. Hiis indicates that the correspondmg bit of the auxiliary data signal was a 
•0" when these samples were encoded. (A "O" at the auxiliary data input of logic circuit 46 
caused it to produce an inverted version of the corresponding PRN datum as its output datum, 
iWng in .subtraction of die corresponding PRN datum from the voice signal) 



resun 



25 Conversely, if subtracting Ae sparse PRN data from the composite data reduces its 

entropy, then the encoding process must have earlier added this noise. TTiis indicates that the 
value of the auxiliary data bit was a T when samples 1, 481, 961, etc., were encoded. 
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By notittg in which case entropy is lower by (a) adding or (b) subirscdng a sparse set of 
PRN data to/from the composite data, it can be detetmined whether die first bit of the auxiliary 
data is (a) a *'0-, or (b) a '•I * (In real life applfcations, in the presence of various distoitiag 
phenomena, the ccnnpositB signal may be sufficiently oorrupted so that iKhher adding nor 
5 subtracting the sparse PRN data actually reduces entropy. Instead, both opeiatioos will increase 
entropy. In this case, che 'correct" operation can be discerned by observing which operation 
increases the entropy less.) 

The foregoing operations can then be conducted for the group of spaced samples of the 
composite data begirming with the second sampie (i.e. 2, 482, 962, ...). The entropy of the 
10 resulting signals indicate whether the second bit of the auxiliaiy data signal is a or a "1 
Likewise with the following 478 groups of spaced samples in die composite s^nal, untii all 4S0 
bits of the code word have been discerned 

As discussed abovc» coirelarion between the composite data signal and the PRN data can 
also be used as a statistical detection technique. Such operations are feciiitoCed m the present 

IS context smce the auxiliary data whose encoded representation is sou^ is known, at least in large 
part, a priori. (In one femi of the technology, the auxiliary data is based on the autbenticatiott 
data exchanged at call set-up, which the cellular system has already received and logged; in 
anodier foini (detailed bdowX the auxiliary data comprises a prcdctenniacd message,) Thus, the 
problem can be reduced to determining wh^er an expected signal is present or not (rather than 

20 lookmg for an entirely unknown signal). Moreover, dau formatter 2020 breaks the composite 
daJa into fiames of known length. (In a known GSM irapleinaitation, voice daU is sent m time 
slots which convey 1 14 data bits each.) By padding the auxiliary data as necessary, each 
repetition of the auxiliary data can be made to start, eg., at the beginning of such a frame of 
data. This, too, simplifies die correlation determinations, since 113 of every 114 possible bit 

25 alignments can be ignored (facilitating decoding even if none of the auxiliary data is known a 
priori). 

Again, this wirdess fraud detection poses the now-familiar problem of d^ecting known 
signals in noise, and the approaches discussed earlier are equally applicable here. 
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Where, as here, the location of the auxiliary signal is known a priori (or more accurately, 
known to M within ooe of a few discrete locations, as discussed aboveX then the matdied filter 
approach can often be reduced to a simple vector dot product between a set of sparse PRN data, 
and mean-removed excerpts of the composite signal corrcspcading thereto. (Note that the PRN 

5 data need not be sparse and may arrive in contiguous bursts* such as in British patent publication 
2,196,167 mentioned eariicr wherein a given bit in a message has contiguous PRN values 
associated widi it.) Such a process st^s through all 480 sparse sets of PRN data and performs 
corresponding dot product operations. If the dot product is positive, the corresponding bit of the 
auxiliajy data signal is a " I if the dot product is negative, the cofrcsponding bit of the auxiliary 

10 data signal is a "0." If several alignments of the auxiliary data signal within the ftamcd 

composite signal are possible, this procedure is repeated at each candidate alignment, and the one 
yielding the highest correlation is taken as true. (Once the correct alignment is determined for a 
single bit of the auxiliary data signal, the alignment of ail the otiier bits can be determined 
therc&om, **Alignmem,'* perhaps better known as "synchronization," can be achieved by 

15 primarily through the very same mechanisms which lock on and track the voice signal itself and 
allow for the basic fiinctioning of the cellular unit). 



Security Considerations 

Secariiy of die presently described cmbodunents depends, ia large part, on security of die 
20 PRN data and/or security of the auxiliary data. In the Mowing discussion, a few of many 
possible techniques fa assuring the security of diese data are discussed. 

In a 6ist embodiment, each teiepboae 2010 is provided with a long noise key unique to 
the telephone. This key may be, e.g^ a highly umque 10,000 bit string stored in ROM. (In most 
^^jpllcations^ keys substantially shorter than this may be used.) 

25 The central office 20 1 4 has access to a secure disk 2052 ot which swJi key data for all 

authoriffid telephones are stored. (The disk may be remote from the office itself.) 
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Each time the telephone is us^d, fifty bits from this noise key are identified and used as 
die seed for a deteiministic pseudo random number generator. The data generated by this PRN 
genezBtor serve as the PRN data for diat telephone call. 

The fifty bit seed can be determined, e.g., by using a random number generator in die 
5 telephone to generate an offiet address between 0 and 9,950 each t&ne the telephone is used to 
place a call. The fifty bits in die noise key beginning at this address are used as the seed. 

During caU setup, this o£&et address is transmitted by the telephone, dxrough the cell site 
2012, to the central office 2014. There, a computer at die cenirai office uses the offsec address to 
index its copy of die noise key for that telephone. The central ofBce diereby identifies die same 
10 50 bit seed as was identified at die telephone. The central of&ce 2014 then xldys these 50 bits to 
die cell site 2012, where a detenoinisdc noise generator like that in the telephone generates a 
PRN sequence corresponding to die 50 bit key and applies same to its decoder 2038. 

By the foregoing process, tiie same sequence of PRN data is generated both at the 
telephone and at the cell site. Accordingly, the auxiliary data encoded on die voice data by the 
15 telephone can be securely Hansmitted to, and accurately decoded by, die cell site. If diis auxiliary 
data does not match die expected auxiliary data (eg. data transmitted at call set-upX die call is 
flagged as fraudulent and appropriate remedial action is taken. 

It will be recognized that an eavesdropper listening to radio transmission of call set-up 
infomiation can intercept only the randomly generated ofi&et address transmitted by the telephone 
20 to die cell site. lUs data, alone» is useless in pirating calls. Even if the hacker had access to die 
signals provided fiom die centtai office to die cell site, diis data too is essentially useless: all diat 
is provided is a 50 bit seed. Since this seed is difierent for nearly each call (repeating only 1 out 
of every 9,950 calls), it too is unavailing to the hacker. 

In a related system, the entire 10,000 bit noise key can be used as a seed. An offset 
• 25 address randomly generated by die telephone durii^ call set-up can be used to identify where, in 
the PRN data lesultmg fiom diat seed, die PRN data to be used for diat session is to begin. 
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(Assunting 4800 voice samples per seccmd, 4S00 PRN data are required per secoad, or about 17 
million PRN data per hour. Accoidingiy, the offset address in this variant embodiment will likely 
be far larger than the offiet address described above.) 

In this variant embodiment the PRN data used for decoding is prefbably genexaiied at she 
5 central stadon from dxe 10,000 bit seed, and relayed to the cell site. (For security reasons, the 
lOtOOO bit noise key should not leave the security of die cencraJ office.) 

In variants of the foregoing systems, die o&et address can be generated by the central 
station or at the cell site, and relayed to the telephone during call set-up, ntther than vice versa. 

In another embodiment, the telephone 2010 may be provided with a list of one-time 
10 seeds, matching a list of seeds stored on the secure disk 2052 at the central office. Each time the 
telephone is used to originate a new call, the next seed in the list is used. By this arrangement^ 
no data needs to be exchanged relating to the seed; the telephone and the carrier each 
independently know which seed to use to generate the pseudo random data seqiience for the 
cumsnt session. 



15 In such an embodiment, the carrier can determine when the telephone has neariy 

exhausted its list of seeds, and can transmit a substitute list (e.g. as part of administralive data 
occasionally provided to the telephone). To enhance security, the carrier may require that the 
telephcme be returned for manual reprograrmning, to avoid radio transmission of this sensitive 
mformation. Alternatively, the substitute seed list can be encrypted for radio transmission using 

20 any of a varieqr of well known techniques. 

In a second class of embodiments, security derives not from die security of the PRN data, 
but from security of the auxiliary message data encoded thereby. One such system relies on 
nansmissktn of a randomly selected one of 256 possible messages. 

In this embodiment, a ROM m die telephone stores 256 diSbcnt messages (each message 
25 may be. eg., 128 bits in length). When the telephone is operated to mitiate a call, the telephone 
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randomly generates a number bctwctti 1 and 256, which serves as an index to these stored 
messages. This index zs transmitted to the cell site during call set-up, allowing the central station 
to identify the expeaed message from a matching database on secure disk 2052 containing the 
same 256 messages. (Each telephone has a dififercnt collection of messages.) (Alternatively, the 

5 carrier may randomly select the index number dtmng call set-up and transmit it to the teicpiione, 
identifying the message co be used during that session.) in a tfaeoraically pore world where 
proposed attacks to a secure system are only mathematical in nature, much of these additional 
hym of security might seem superfluous. (The addition of these extra layers of security, such as 
differing the messages themselves, simply adoxowledge that the designer of actual pubtic- 

10 functioning secure systems will face certain implementation economics which might compromise 
the mathematical security of the cone principles of diis technology, and thus these auxiliary layers 
of security may afford new toob against die inevitable attacks on unplementation). 

Thereafter, all voice data transmitted by the telephone for die duration of that call is 
steganographically encoded with die indexed message. The cell site checks the data received 
15 from the telephone &r the presence of the expected message. If the message is absent^ or if a 
different message is decoded instead, the call is flagged as fraudulent and remedial acdcni is 
taken. 

In this second embodiment, the PRN data used for encoding and decoding can be as 
simple or complex as desired. A simple system may use the same PRN data for each call. Such 

20 data may be generated, e.g.» by a deterniinistic PRN generator seeded with fixed data unique to 
Che telephone and known also by the central stadon (e.g. a telephone identi&rX or a universal 
noise sequence can be used (i.e. the same noise sequence can be used for all telephones). Or the 
pseudo random data can be generated by a detennmistic PRN generator seeded with data that 
changes from call to call (e.g. based on data transmitted during call set-up identifying, eg., die 

25 destination telqriione number, etc.). Some embodiments may seed the pseudo random number 
geneiaxa^ with daia from a preceding call (since this data is necessarily known to the telephone 
and the carrier, but is likely not known to pirates). 

Naturally, elements from the foregoing two approaches can be combined in various ways, 
and supplemented by other fieatures. The foregoing embodiments are exemplary only, and do not 
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begin to catalog the myriad approaches which may be used. Generally speaking, any data which 
is neeessariiy known or knowable by both the telephone and the ceil site/central statiotu can be 
used as the basis for either the auxiliary message data, or the PRN data by which it is encoded. 



Since the preferred emimdiments of this aspect of die present technobgy each redundandy 
5 encodes the auxiliary data throughout the duration of the subscriber's digitized voice, the auxiliary 
data can be decoded from any brief sample of received audio. In die preferred forms of this 
aspect of the technology, the carrier repeatedly checks die st^anographically encoded auxiliary 
data (c.g, every 10 seconds, or at random intervals) to assuie that it continues to have die 
expected attributes. 



10 While die foregoing discussion has focused on steganographically encoding a transmission 

from a cellular telephone, it will be recognized that transmissions to a ceUular telephone can be 
steganographically encoded as well. Such arrangements find applicability, e.g., in conveying 
administrative data (Le. non-voke data) from the canier to individnal telephones. This 
administrative data can be used, for example, to reprogram parameters of targeted ceUular 

13 telephones ((ff all ceUular telephones) from a central tocation, to update seed lists (for systems 
employing die above^ribed on-time pad systemX to apprise Toamasg" cellular telephones of 
data unique to an unfamiliar bcal area. etc. 



In some embodiments, the carrier may steganographically transmit to die cellular 
telephone a seed which the ceUular phone is to use in its transmissions to die carrier during die 
20 remainder of that session. 

While die-foregomg discussion has focused on stegaoographic encoding of the baseband 
dighized voice data, artisans will recognize that intermediate frequency signals (whedier analog or 
digitaO can likewise be steganographically encoded in accordance with principles of the 
technology. An advani^ of post-baseband encoding is that die bandwiddi of diesc intermediate 
25 sigmds is rchuively large compared widi die baseband signal, allowing more auxiliary data to be 
encoded dierein, or allowing a fixed amount of auxiliary data to be repealed more frequently 
durmg transmission. (If steganogr^hic encoding of an intermediate signal is employed, care 
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should be taken that the peiturbations introduced by the encoding are not so targe as to interfere 
with icJiabie transmisston of the administzative d3ta» taking into account any eiror conectmg 
fiictlities sixppoftcd by the packet format). 

Those skilled in the ait will recognize diat the auxiliary data, itself, can be arranged in 
5 known ways to support error detecting, or error correcting capabilities by the decoder 3S. The 
hiteitsted reader is referred, e.g., to Rorabaugh, Error Coding Cookbook McGraw Hill, 1996, one 
of many readily available texts detailing such techniques. 

While the preferred embodiment of this aspect of the technotogy is illustrated in the 
context of a cellular system utilizing packetizcd data, other wireless systems do not employ such 

10 conveniently framed data. In systems in which framing is not available as an aid to 

synchronization, synchionization marking can be adiieved within the conposite data signal by 
techniques soch as that detailed In applicant's prior applications. En one class of such techniques, 
the auxiliary data itself has characteristics facilitating its synchronization. In anodier class of 
techniques, the auxiliary data modulates one or more embedded carrier patterns which are 

15 designed to &cilitate alignment and detection. 

As noted earlier, the principles of the technology arc not restricted to use vrith the 
particular forms of st^anographic encoding detailed above. Indeed, any steganographic encoding 
technique previously known, or hereafter mvented, can be used in the fashion detailed above to 
enhance the security or fimctionality of cellular (or other wireless, e.g. PCS) communications 
20 systems. Likewise, these principles are not restricted to wireless telepAones; any wireless 
transmission may be provided with an "in-band" channel of this type. 

It will be recognized tiiat systems for implementing applicant's technology can comprises 
dedicated hardware circuit elements, but more commonly comprise suitably programmed 
microprocessors wirii associated RAM and ROM memory (e.g. one such system in each of the 
23 tdepfaone 2010, cell-site 2012, and central office 2014). 
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FMf;nr>>wr, BY BIT CELLS 



The foregoii^ discusskms have fixused on maementing or decFememtag the values of 
individual piuls. or of groups of pixels (bumps), to reflect encoding of an auxiliary data signal 
combmed with a pseudo random noise signal. TTib following discussion details a variant 
S embodiment wfaeiein tl» auxiliaiy data - wiihoitf pseudo landomiration - is encoded by 
patterned groups of pixels, here termed bit ceils. 



Re&iTing to Figs 41A and 41B, two illustrative 2x2 bit cells are shown. Fig. 41A is used 
to represent a '0" bit of the auxiliary data, while Fig. 41B is used to represent a "1" bit. In 
operation, the pixels of the underlying image are tweaiced op or down in accordance with the +/- 
10 values of die bit cells to lepiesent one of these two bit values. (ITie magnitude of die tweaking at 
any given pixel, bit ceil or region of die image can be a fimction of many fecton, as detailed 
betaw. It is the sign (rf die tweaking that defines the characteristic pattern.) In decoding, the 
relative biases of the encoded pixels are examined (using techniques described above) to identify, 
for each comspomiing region of the encoded image, which of the two patterns is represented. 



15 White the auxiliary data is not explicitly randomized in diis embodiment, it will be 

reeogniaed diat die bit cell patterns may be viewed as a "designed" carrier signal, as discussed 
above. 



The substiwtion of the previous embodiments' pseudo taadom noise widi die present 
-designed" infonnalioa earner affotds an advantage: die bit cell paimmins manifists itself in 
20 Fourier space. Thus, the bit cell patterning can act like the subliminal digital graticules discussed 
above to help register a suspect image to remove scale/iotation enors. By changing the size of 
the bit cell, and the pattern therein, the location of the energy thereby produced in die spatial 
traasfoim domain can be tailored to optimize independence from typical imagery energy and 
gcilitate detection. 



SUBSTITUTE SHEET (RULE 26) 



WOy7/43736 



PCT/US97/0R351 . 



. 124^- 

(While the foregoing discussioQ contemplates that the auxiliary data is encoded directly 
without raodomization by a PRN signal^ in other embodiments, randomization can of course be 
used) 



.5 ypRK QN PERCEPTUALLY ADAPTTVE SlGNINj3 

In several of the above-detailed embodiments, the magnimde of die signature energy was 
tailored on a region*by-region basb to rsduce its visibility in an image (or its audibility in a 
sound, etc). In the following discussion, ^pUcant more particularly coosideis die issue of hiding 
signature ene^ in an im^e, die separate issues thereby posed, and solutions to each of diese 
10 issues. 

The goal of the signing process, beyond simpiy functioning, is to maximize the "numeric 
detectability" of an embedded signanire while meeting some fonn of fixed "visifaility/acceptafaili^ 
threshold" set by a given user/creator. 

In savicc to design toward diis goal, imagine die following three axis param^r space, 
15 where two of the axes are only half-axes (positive onlyX and die third is a fall axis (botfi negative 
and positive). This set of axes define two of die usual eight octal spaces of euclidcan 3-spaoc 
As tilings icfine and "deservedly separable" parameters show up on the scene (such as ''extended 
local visibility mctrics"X then they can define dieir own (generally), half-axis and extend die 
following example beyond three dimensions. 

20 The signing design goal becomes optimally assigning a "gain" to a local bump based on 

its coordmatcs in die above defined space, whilst keeping m mind die basic needs of domg die 
operations fest in real applications. To begin with, die dirce axes are die following. We'll call 
the two half axes X and y, while the fall axis will be z. 
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Hie X axis repitsents the luminance of the singuiar bump. The basic idea is that you can 
squeeze a little mote energy into bright regions as opposed to dim ones. It is important to note 
that when trae "pqrcho-iinear • device independent" luminance values (pixel DN*s) come along, 
this axis might become superSuous, unless of course if the luminance value couples into die other 
5 operative axes (e.g. C*xy). For now, this is here as much due to the sidH)pttmali9 of current 
quasi-linear luminance coding. 



The y axis is the "local hiding potential" of die neighborhood within which the bump 
finds itself. The basic idea is that flat regions have a low hidmg potential since the eye can 
detect subtle changes in such regions, whereas complex textured regions have a high hiding 

10 potential. Long lines and long edges tend toward die lower hiding potential since ^'breaks and 
choppines" in nice smooth long lines are also somewhat visibte, while shorter lines and edges, 
and mosaics thereof, tend toward the higher hiding potential. The« latter notions of long and 
short are directly connected to processing time issncs, as well to issues of die engineering 
resowces needed to carefully quantify such parameters. Developing the working model of the 

15 y-axis will mcvitably entail one part theory to one part picky-aitist-cmpiricism. As the parts of 
the hodge-podge y-axis become better known, they can splinter off into their own independent 
axesif it's worth it. 



20 



The z-axis is die *Vith or against die grain" (discussed below) axis which is die faU axis - 
as opposed to d;e other two half-axes. The basic idea is that a given input bump has a 
pre-existing bias relative to wAedier one wishes to encode a M' or a at its location, which to 
some non-tiivial extent is a fimction of the readmg algorithms which will be employed, whose 
(Was) magnitude is semi-conelatcd to die "hidmg potential" of die y-axis, and, fiatunately, can be 
used advantageously as a variable m dctennining what magnftude of a tweak vahie is assigned to 
die bump m question. The concomitant basic idea is that when a bump is already your friend 
25 (t-e. its bias relative lo its neighbore ahxady tends towards the desired delta value), then don't 
change it much. Its natural state already provides the delta energy needed for decoding, without 
altering the localized hnage value much, if at all. Conversely, if a bump is initiaUy your enemy 
(Le. its bias relative to its neighbocs tends away from the delta sought to be imposed by die 
encoding), then change it an exaggerated amount. This later operadon tends to reduce the 
30 excursion of this point relative to its neighbors, making die point less visibly conspicuous (a 
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hi^ly localized bluxring operation), while providing additional energy detectable when decoding. 
These two cases are termed "with the gram** and 'against the grain" herein. 

The aix)ve general description of the prafaiem should sufBce for many years. Clearly 
addmg in chrominance issues will expand the definitions a bit, Jeading lo a hii more signature 
5 bang for die visibility, and human visibility research which is applied to the problem of 

compression can equally be applied to this area but for diametrically opposed reasons. Here are 
guiding principles which can be employed in an exemplary application. 

For speed^s sake» local hiding potential can be calculated csnly based on a 3 by 3 
neighboriiood of pixels, the center one being signed and its eight neighbor. B^ond speed 
10 issues, there is also no data or coherent theoiv to suppoit ai^thing larger as weil. The design 
issue boils down to canning the y-axis visibility thing, how to couple the luminance into this, and 
a little bit on the fiiend/enemy asymmetry thing. A guiding principle is to simply make a flat 
region zero, a classic pure maxima or minima region a ''1. 0" or the highest value, and to have 
"local lines", ^'smooth slopes", "saddle points'' and whatnot &11 cut somewhere m between. 

15 The exemplary application uses six basic parameters: 1) lummaoce; 2} difference from 

local average; 3) the asymmetry fiictor (with or against the grain); 4) minimum linear &ctor (our 
crude attempt at flat v. lines v. maxima); S) bit plane bias factor; and 6) j^obal gain (the user's 
single top level gain knob). 

The Luminance, and Difference from Local Avoage parameters are soai^ forward, and 
20 dieir use is addressed elsewhere m this specification. 

The Asymmetry factor is a single scalar applied U> the "against the grain" side of the 
difference axis of number 2 directly above. 

The Minimum Linear fiictor is admittedly crude but it should be of some s«vice even in 
a 3 by 3 neighborhood setting. The idea is that tme 2D local minima and maxima will be highly 
25 perturbed along each of the four lines travelling through the center pixel of the 3 by 3 
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neighborhood, while a visual line or edge will tend k> flatten out at least one of the four linear 
profiles. [The four linear profiles are each 3 pixels in length, i.e., the top left pixel - center - 
bottom ri^ the top center • center - bottom centen the top right - center - bottom left; the right 
center - center - left center,]. Let's choose some moric of ennopy as applied to three pixels in a 
5 row, perform this on all four linear profiles^ then choose the mioimum value for our ultunate 
parameter to be used as our y-^s.^ 

The Bit Plane Bias factor is an interesting creature with two faces, the pre-emptive face 
and die post-emptive fact. In the former, you simply "read" the unsigned image and see where 
all the biases ftdl out for all the bit planes, then simply boost the "global gain" of the bit planes 

10 which are, in total, going against your desired message, and leave the others alone or even 
slightly lower their gain. In the post-emptive case, you chum out the whole signing process 
replete widi the pre-emptive bit plane bias and die other 5 parameters listed here, and then you, 
e.g^ run the signed onage through heavy JPEG compression AND model the ''gestait distoition" 
of line screen printxcg and subsequent scanning of the image, and then you read the image and 

15 find out which bit planes are struggling or even in error, you appropriately beef up the bit plane 
bias, and you run through die process again. If you have good data driving the beefing process 
you should only need to perfonn this step once, or, you can easily Van-Cittertize the process 
(arcane refmnce to reiterate the process with some damping factor applied to die tweaks). 

Finally, there is the Global Gam. The goal is to make this single variable the top level 
20 "intensity knob" (more typically a slider or odier control on a graphical user interface) that the 
slightly curious user can adjust if ±ey want to. The veiy curious user can navigate down 
advanced menus to get their experimental hands on the other five variables here, as well as 
others. 



25 Visible Watemiark 
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la ceiuin applications it is desirable to apply a visible indicia to an image co indicate that 
it includes snegancgn^iucally encoded data. In one embodiment, diis indicia can be a lightly 
visible logo (sometimes termed a "watennark'*) applied to one comer of the hn^. This indicates 
that the inu^ is a ''sman" image* conveying data in addition to the imagery. A lightbulb is one 
5 suitable bgo. 
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If marking of images becomes widespread (e.g^ by software comparible wirfi Adobe's 
image processing software), a user of such software can decode the embedded data fiom an image 
and consult a public registry to identify the proprietor of the image. In some embodiments, the 
registry can serve as the conduit through which appropriaie royalty paymeozs are forwarded to die 
proprittor for the user's use of an imstt;e. (In an iifaisiraxive embodiment the registry is a server 
on the Internet, accessible via die Wwld Wide Web, coupled lo a database. The dai^asc 
includes detailed information on catalogued images (e^^ name, address, phone number of 
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proprietor, and a schedule of charges for different types of uses to whicb the image may be putX 
mdexed by identificalion codes with which the images themselves azt encoded A person who 
decodes an image queries the registry widi the codes thereby gleaned to obtain the desired data 
and, if appropriate, to ft>rwani electronic payment of a copyright royalty to the image's 
5 proprietor.) 



Particular Daa Formats 

While the foregoing steganography techniques are broadly applicable, dieir coimnerciai 
acceptance will be aided by establishment of standards setting forth which pixels/bit ceils 
10 represent what. The following discussion proposes one set of poss3>le standards* For expository 
convenience, this discussion focuses on decoding of the data; encoding follows in a 
straightforward manner. 

Referring to Fig. 42, an im^ 1202 includes a plurality of tiled ''signature blocks" 1204. 
(Partial signature blocks may be presort at the image edges.) Each signamre block 1204 includes 
15 an 8 X 8 anay of sub-biocks 1206. Each sub-block 1206 inchides an 8 & 8 array of bit cells 
1208, Each bit cell comprises a 2 x 2 anay of "bumps" 1210, Each bump 1210, in nnm, 
comprises a square grouping of 16 individual pucels 1212. 

The individual pixels 1212 are the smallest quanta of image data. In this arrangement, 
however, pixel values ate not, individually, the data carrying elements. InsBad, this role is served 
20 by bit cells 120ff (i,e. 2x2 arrays of bumps 1210). In particular, the bumps comprising the bits 
celts are encoded to assume one of the two patterns shown in Fig. 41. As noted earlier, the 
pattern shown m Fig. 41A represents a "0" bit, while die pattern shown in Fig. 41B represents a 
T bit Each bit cell 1208 (64 image pixels) thus represents a single bit of the embedded data. 
FafTh sub-block 1206 mcludes 64 fait cells, and thus conveys 64 bits of embedded data. 
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(The nanira of the image charges effected by the encoding follows the techniques set 
forth above under the heading MORE ON PERCEPTUALLY ADAPTIVE SIGNING; that 
discussion is not rqieated here.) 



In the illustrated embodiment, the embedded data inciudes two parts: control bits and 
5 message bits. Use 16 bit cells 1208A in the center of each sub-block 1206 serve to convey 16 
control bits. The surrounding 4J bit cells I2MB serve to convey 48 message bits. This 64-bit 
chunk of Hafa is encoded in each of die sub-blocks 1206» and is repeated 64 times in each 
signature block 1204. 



A digression: in addition to encoding of the image to redundantly embed the 64 
10 control/message bits therein, the values of mdividual pixels arc additionally adjusted to effect 
encoding of subliminal gtaticules through Ae image. In this embodiment, the graticules discussed 
m conjunction with Fig. 29A are used, resulting in an unperceptible texturing of the image. 
When the image is to be decoded, the image is tiansfenncd into the spatial domain, the Fourier- 
Mdlin technique is appBcd to match the graticule energy points with their expected positions, and 
15 the processed data is dien invcise-iransfoimed, providing a roistered image ready for decoding. 
(The sequence of fir^t tweakmg the image to effect encoding of die subliminal graticules, or first 
tweaking the image to efifect encoding of the embedded data, is not believed to be critical. As 
presently practiced, the local gain facion (discussed above) arc computed: then the data is 
encoded; then die subliminal graticule encoding is performed. (Bodi of these encoding steps 
20 make use of the local gain factors.)) 

Retaming to the data fomat, once the encoded image has been thus registered, the 
locations of the control bits in sub-block 1206 are known» The image is then analyzed, *m the 
aggregate considering the "northwesteni-mosf sub-block 1206 from each signature block 
1204), to detwmine the vahie of control bit Ml (represented in sub-block 1206 by bit cell 
25 1208Aa). If this vahie is determined (eg, by statistical techniques of die sort detailed above) to 
be a "I," this indicates th^ the format of the embedded data conforms tn the standard detailed 
herem (the Digmiarc Beta Data Format). According to this standard, control bit #2 (represented 
by bit cells I208Ab) is a flag indicating whether the image is copyrighted. Control bit #3 



SUBSTITUTE SHEET (RULE 26) 



wo 97/43736 



PCT/DS97/08351 



. 132 

(represented by bit cells I208Ac) is a flag indicating whether the image is unsuitable for viewing 
by children. Certain of the remaining bits are used for error detection/coirection purposes. 

The 4S message bits of each sub block 1206 can be put to any use; they are not specified 
in this foimat One possible use is to define a numeric "owner" field and a numeric ''image/item" 
5 field (€.g. 24 bits each). 

If this data foimat is used, each sub-block 1206 contains the entire control/message data, 
so same is repeated 64 times widiin each signature block of the image. 

If control bic #1 is not a then the foimat of the embedded data does not ccxifonn to 
the above dccribed standard. In this case, the reading sofbrare analyzes the image data to 
10 determine the value of control bit #4. [f this bit is set (i.e. equal to "1"), this signifies an 
embedded ASCII message. The reading software then exammes control bits ^5 and #6 to 
determine the length of the embedded ASCII message. 

If control bits #5 and #6 bodi are "O," diis indicates the ASCII message is 6 characters in 
length. In diis case, the 48 bit cells I208B sunounding the control bits I208A are interpreted as 
15 six ASCn characters (8 bits each). Again, each sub-block 1206 contains the entire 

contiol/message data, so same is repeated 64 times widiin each signature block 1204 of die 
image. 

If control bit #5 is '•Q" and control bit U6 is "I," this indicates the embedded ASCII 
message is 14 characten in lengtfi. In this case, the 48 bit cells 1208B surrounding the control 
20 bits 1208A are intepreied as the first six ASCII chaiaders. The 64 bit cells 1208 of the 
immediately-adjoining sub-block 1220 are i iitta p rer ed as the final eight ASCII characters. 

Note that in this arrangement, the bii-celis 1208 in the center of sub-block 1220 are not 
interpreted as control bits. Instead, the entire sub-block serves to convey additional message bits. 
In this case diere is just one group of control bits for two sub-blocks. 
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Also note than in this aiiangement, pairs of sub-blocks 1206 contains the entire 
c<mm>i/message data, so same is repeated 32 times within each signamre block 1204 of the 
image. 

Likewise if control bit #5 is T and control bit #6 is "0." This indicates the embedded 
S ASCII message is 30 characters in length. In this case, 2x2 arrays of sub-blocks are used for 
each repiesentadon of Ae data. The 48 bit cells I20SB surrounding control bits 1208A are 
interpreted as the first sue ASCII characters. The 64 bit cells of each of adjoining block 1220 arc 
interpret as lepresenting the 8 additional characters. The 64 bits cells of sub-block 1222 
are interpreted as representing the next 8 characters. And the 64 bit cells of sub-block 1224 are 
10 interpretEd as representing the fmal 8 characters. In tfiis case, there is just one group of control 
bits for ^ir sub-blocks. And the control/message data is repeated 16 times within each signature 
block 1204 of die image. 

If control bits U5 and #6 are both "Ts, this indicates an ASCII message of programmable 
length. In this case, the reading software examines the first 16 bit cells 1208B surrounding the 
' 15 control bits. Instead of interpreting these bit cells as message bits, they arc interpreted as 

additionai control bits (the opposite of the case described above, where bit cells normally used to 
represent conliol bits rqaresentcd message bits instead). In particular, the reading software 
interprets diese 16 bits as rcprcscndng, in binary, the leagdi of the ASCII message. An algorithm 
is then applied to this data (matchii^ a similar algorithm used during the encoding process) to 
20 establish a corresponding tiling pattern {le. to specify which sub-blocks convey which bits of the 
ASCn messi^e, and which convey control bits.) 

In diis programmable-length ASCII message case, control bits are desirably repeated 
several times widiin a single representation of die message so that, e.g., there is one set of control 
bits for approximately every 24 ASCII characters. To increase packing efficiency, the tiling 
25 algorithm can allocate (divide) a sub-block so that some of its bit-cells are used for a fust 
iqjresentation of the message, and others are used for another representaiion of the message. 
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Refeence was earlier made to begioning the decoding of die roistered image by 
considering the 'noithwestera-most" sub-block 1206 in each signature biock 1204. This bears 
eiaboratkm. 

Depending on the data format used, some of the sub-blocks 1206 in each signature bfock 
5 1204 may not mchide control bits. Accordingly, the decoding software desirably determines the 
data &nnat by first examining the "nordiweatem-most" sub-block 1206 in each signature block 
1204; the 16 bits cells in the centers of these sub-blocks will reliably represent control bits. 
Based on the value(s) of one or more of these bits (e.g. the Digimarc BeU Data Fonnat bit), the 
decoding software can identify all other locations throughout each signature block 1204 where the 
iO control bits are also encoded (e,g. ai the center of each of the 64 sub4)locks 1206 comprismg a 
sigpatnre block 1204), and can use the larger statistical base of daU diercby provided to extract 
die remaining control bits from the image (and to confirm, if desired, the earUer control bit(s) 
d^rmination). Ajfte- ail contn)! bits have thereby been discerned, the decoding software 
detcmincs (from the control bits) the mapping of message bits to bit cells throughout the unage. 



To reduce the likelihood of visual arti&cis, the numbering of bit cells widiin sub-blocks is 
alternated in a checkerboard-like fashion. That is. the "northwestem-mcst" bit ceU m the 
"northwestem-most" sub-block is numbered "0.'' Numbering increases left to right, and 
successively through the rows, up to bit ceU 63. Each sub-block diametrically adjoinmg one of 
its comers {i.e. sub-block 1224) has the same ordering of bit ceils. But sub-blocks adjoining its 
edges (i.e, sub-bkscks 1220 and 1222) have the opposite numbering. That is, the "nordwcstan- 
most" bit cell in sub-blocks 1220 and 1222 is numbered "63." Numbering decreases left to right, 
and successively through the rows, down to 0. Likewise tfnoughout each signamre block 1204. 



In a variant of the Digimarc beta format, a Ban; of sub-biocks is used for each 
representation of the data, providing 128 bit cells. The center 16 bit cells 1208 m the first sub- 
25 block 1206 are used to represent control bits. The 48 remaining bit cells in that sub-block, 

together wiA all 64 bit cells 1208 in the adjoining sub-block 1220. are used to provide a 1 12-bit 
message field. Lik^wvise for every pair of sub-blocks throughout each signature block 1204. In 
such an arrangement, each signature block 1204 thus includes 32 complete representations of the 
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esKoded data (as opposed to 64 rqsresentarioos in die eariier-described socubrd). This additionaJ 
ieagth allows encoding of longer data strings, such as a maneiic [P address (eg. URL). 

Obviotisfy, numexous alternative dau fonnats can be designed. The particular format 
used can be indicated to the decoding software by values of one or more control bits in the 
5 encoded image. 



Other Applications 

* Before oonciodn^, it be instnictive to review some of fte other fields where 
prindpies of applicant's technology can be emplc3yed. 

15 One is smart business cards, wherein a business card is provided with a photograph 

having unobtrusive, machine-readible contact data embedded therein. (The same fimction can be 
achieved by rhanging the sui&ce microtopoiogy of the card to embed the data therein.) 

Another promising ^plication is in content reguiation. Television signals, images on the 
internet, and other content sources (audio, image, video, ac) can have data indicating their 
20 "appropriatcaas" (i.c. their rating for sex, violence, suitability for children, etc.) actually 

embedded in die content itself rather than externally assoctated therewith. Television receivers, 
internet surfing software, etc., can discern such appropriattness ratings (c.g. by use of univcnal 
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code decoding) mi can take appropriate action (e.g. not pemtitting vievring of aa image or video, 
or play^back of an audio »ouroeX 

In a single embodiment of the foregoing, the embedded data can have one or more "flag" 
bits, as discussed earilCT. One flag bit can signify ''inappropriate for children." (Others can be. 
5 e.g., "Miis image is copyrighted" or "this image is in the public domain.'') Such flag bits can be 
in a field of control bits distinct fix)m an embedded message, or can - themselves - be the 
message. By examining the state of these flag bits, the decoder software can quickly apprise the 
user of various attributes of the image. 

(As discussed eariier* control bits be encoded in known locations in the image — 
10 known relative to the subiiminai graticules - and can indicate the format of the embedded data 
(e,g. its length, its type, etc.) As such, these control bits are analogous to data sometimes 
conveyed in prior art file headers, but in this case they are embedded within an image, instead of 
piepended to a file.) 

The field of meidiandisc marking is generally well served by familiar bar codes and 
15 universal product codes. However, in certain applications, such bar codes are undesirable (e.g. 
fcH* aesthetic considerations, or where security is a concern). In such applications, applicant's 
technology may be used to mark merchandise; either through in innocuous carrier (e.g. a 
photograph associated with the product), or by encoding the microtopology of the merchandise's 
surface, or a label thereon. 

20 There are applications - too numerous to detail - in whkh steganography can 

advantageously be combined with encryption and/or digital signature technology to provide 
enhanced security. 

Medical records appear to be an area in which authentication ts important 
Steganogiaphic principles — applied either to film-based records or to the microtopology of 
25 documents - can be employed to provide some protection against tampering. 
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Many indusuies, e.g. automobile and airline, rely on lags to mark criticai parts. Such 
tags, however, are easily removed, and can often be oounterfeited^ In applications wherein better 
security is desired, indastrial parts can be steganograpbicaJIy marked to provide an inconspicuous 
identificaiion/autbentication tag. 

i In various of the applications reviewed in this specification, diifeient messages can be 

steganogiaphkally conveyed by different regions of an image (eg. dififerent regions of an image 
can provide diffcxent internet URLs, or diffacnt regions of a photocoilage can identify different 
photographers). Likewise with other media (e.g. sound). 

Some software visionaries look to die day when daia blobs will roam the datawaves and 
10 interact with other data blobs. In such an era, it will be necessary for such blobs to have robust 
and incorruptible ways of identifying diemselves. St^anographic techniques again hold much 
promise here. 

Finally, message changmg codes - recursive systems in which steganographicaily 
encoded messages acnially change underlying steganographic code patterns - offer new levels of 
15 sophistication and security. Such message changing codes are particulariy well suited to 

^plications such as plastic cash cards where time-changing elements are impoitant to enhance 
securiQT. 

Again, while applicant prefias the particular forms of steganographic encodmg detailed 
above, the diverse applications disclosed in this specification can largely be practiced with other 
20 steganographic marking techniques, many of which are known in die prior an. Likewise, while 
the specification has focused on applications of diis technology to images, the principles diercof 
are generally equally applicable to embedding such information m audio, physical media, or any 
odier carrier of mfomiatioa. 

Finally, while the specification has been ilhistrated widi particular embodiments, it will be 
25 recognized that elements, components and steps from these embodhnents can be recombined in 
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differeat arrangements to serve different needs and a^iications, as will be readily 
^parent to those of orduiary skill in the art 

5 POINTER TO A HEADER. AND/OR, POINTER TO A PROCESS 

The basic philosophy suxTDunding the use of a "leadei" in a digital file has 
already been discussed, along wiA the idea that steganographicaliy encoded information 
can serve many of diese Auctions independently or in parallel with a traditional header. 

10 This section anempts to farther clarify this notion as well as further clarify the notion 
that att ^c^ information can act as a "^pointef to a remote database cisitaining the 
information that might be otherwise contained in a header. Refer to the sections on 
header verification, the *1]odier^ and netw<^ linidng methods and hot objects in 
general. This section also expands upon the idea that header information and other 

15 kinds of attached information can be more than simply passive descriptive information* 
but can be fiilly execatable code or other forms of ''acuve information and network 
links." 

As previously developed in the section on hot otgccts, given that realtime connection of 
a desktop computer, a television set or a stero system onto a network is becoming more 

20 and more commonplace, the absolute need for an attached header replete with full 

infonnation about a data object is diminisliing, and the possibility of simply pointing to 
a remote yet connected database is becoming mOTe and more practical. The persistent 
and immediate comicction to the infcsmation is more fundamental tiian strictly needing 
the full information attached to the data object (where time to change informadon 

35 presents a parncnlar problem for object-attached information). 

Referring back to Fig. 1 6, and harkenmg back to the discussion on networked 
linking methods, we find the situation depicted in Fig. 60. The header infonnation 
"joe's image" is now only located on some acces^ble remote database, and the 
infmnation attached to the object in either the actual header^ or steganographicaliy 

30 embedded, ot both, now contains simply an address to the remote database location. 
Software applications would then be manufactured such diat a networked connecdon 
would be assumed to be either acdve or cap^le of becoming so rather swiftly, and that 
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when a data object such as an image or a video clip was read in* the embedded address 
of the header information would be read, a quick retrieve of the database information 
would be performed, and the remote header ioformatton would be used Just as if it were 
locally attached to the data object 

5 It is important to note that the stegonogr^hic embedded infonnation does not 

necessarily need to be present in order to make such a system fonctioa and vice versa, 
the traditional header need not be present if steganogrqphic information is present The 
two together represent a value-adcfed redundancy and greater assurance that the link 
between the remote information and the object remain intact. Likewise, even if thoe 

10 docs exist some **mastcr remote headci^* for a given data object, an "static -in-time copy" 
(by deflnition) of thai master header file can still travel with the data object for the 
purposes of speed or for the purposes of servicing n(^-networked sties or applications, 
with reference back to the master if need be for any latest updates. Likewise, though 
the principles of this application focus largely on audio-visual signals and data objects, 

1$ many of these design principles spply to text files, software programs, basic data 

records, said the like, where steganography per sc may not even be af^licablc. Likewise, 
any other kind of information can c^ainly still be attached to an object in addition to 
the above mentioned "header" information. Another way to look at it is that this new 
''header address** can simply be appended to or incorporated into any existing header 

20 format such as JPEG, TIFF, etc. 

It is also impcfftant to note that the infonnation stored remotely need not simply 
be ''passive hrformation" such as a person's name or the latest {oice for some peddled 
item. It can also be computer instructions or software executable code diat is associe^ 
with the data object Thus a software application can read in a data object reading at 

25 the same tnne the remote database address, retrieve via a network the remote 

connecdoiu then either display the remote information along with the data object 
information itself, or actually launch some local application mudi like JAVA based 
systems currently function, or the remote database can acuially ship executable code 
which may then be invoked locally on the machine where the data object has been read. 

30 In this fashion, a data object becomes a first stepping stone into some computmg 

experience usually rela^ to the content of the object itself, much as a cover of a book 
is a quick visual summary of the book's content where the challenge becomes enticing 
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third parties to enter. 

Thenodofi of a singular string of attached bits refeni^g to aremote database 
wherein a master heador is located we shall hencefordi refer to as an 'Identity pointer^. 
We i ^Hali lefer to the database locaticm containing the roaster information connected 
with that data object as the "'master id^ty.** 

As is well known even in the prior ait of tracUtional mail, addresses can be 
hieraichzcal in nature, with ev^ finer levels of granularity in searching for the ultimate 
central location or master identity. So too vritdi identity pointers, tbey can acnially be a 
sin^e string of hierarchical information parsed according to where an application or 
machine needs to go in order to r^eve the full information associated witii the data 
object and/or audic/visual signal. Thus, a sub-string inside the identity pointer might 
indicate, through proper prograinining of an application or instrument including address 
redirection tables, that the master identity for such and such object exists inside one's 
own computer at such and such directory and m such and such file. On the other hand, 
a first section of the identity pcinter might indicate a search is required m the world 
wide web for a particular rights managanent database or commerce server database, a 
second section of the identity pointer thai "points" to an address at this particular 
database, wherein what is pointed to may be the ultimate location of the master identity, 
or the pointed-to location is actually a database of further pointers to a multitude of 
databases, indicating that a third section of the identity pointer now has an address 
locaiiwi of the ultimo master identity. This formula can clearly recurse further imo 
fourth, fifth, and so on layers. The point is that it is not strictly required to have the 
identity pomter only point to some singular master database, but tiiat there will in fact 
be a muldmde of intax:onnecied databases which will be the rule rather than the 
exception. This includes databases which are more traditionally viewed as belonging in 
some kind of network which is not necessarily as large as the mtemet, such as local area 
networks, or even a personal computers series of hard drives, or whatever. The main 
overall point is that the identity pomter contains sufficient information in order to be 
able to find the master identity, where k is a given that an application or instrument can 
read the master pointer, interpret it, and do the searching and retrieving of tbs master 
identity. 

Those skilled in the art may notice thai so &r in this section no mmnon has been 
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made about encrypting either the identity pointer or the master ideniicy or any 
communications involved in the overall process. Such improvements can certainly be 
made and enhance more of the ^'secure idemification" aspects of an attached 
infoimation system, or die related "verification'' function that many commerce 
6 applications will be seeking. The basic principles of the invention can nevenhdess be 
followed. 

The identity pointer can also point to locations wiiich arc not (currently) viewed 
as strictiy a database. For example^ a data object can contain a sub-string which points 
to a particular application that needs to be launched in order to resolve the remainder of 

10 tiic identity pointer string. A more spccifk example of tlie general example might be 

that a sub-string in the identity pointer instructs a machine to launch a web browser, and 
the remainder of the string would be a traditional URL to some web page containing 
basic contact faiformation to the creator of a video production. Users on a local machine 
might program the machine to their own liking in that they would "allow'* content with 

15 these abilities to go ahead and launch browsers when in^cted, or they can tell their 
local machine to ignore these requests by a givai piece of content. LUcewise, an sub- 
string in the identity pointer might instruct a ranltimcdia database interface program to 
be launched and die location of the object m question sought* found, and displayed. 
It can be readily seen tiiat the general principles of the invention would indicate 

20 that applications and insmnnenis which deal with data objects require less and less local 
intelligence regarding "interpreting and operating upon" attached information* yet 
instead they become better and better at being realtime connected to a broader network 
and know how to properly find and reirieve information from various locations on that 
broader network. Moreover, ^plications and mstruments so enabled would abo make 

2S use of well-known-m-the-art metiiods of launching new execution dueads or entire new 
applications based on the retrieved information. The physical location of the master 
identity, as weU as die "kvcP it exists on in the network hierarchy (from CPU RAM, 
throng Operating System, through local storage disks, through local networks, up 
thiou^ global networks such as the Internet) becomes immaterial so long as an 

30 application or instrument is equi^ed with the realtime abili^ to find and retrieve the 
pertinent information. 

We now cum toward a more detailed description of the drawings. Fig. 60 depicts 
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' the esscmial concept behind the "^master identity object" and a ^stem buih aionnd it 
(""Nfaster Identity Objecf \ or MIO, is here used as a convenicmoe and should not be 
constnied as limiting the kind of ^formation pointed to by an embedded paisisr. It,. 
the MO could easily pcix^ to executable code or execot^le sequences for example 
5 rather than identity infonnatfon). Indeed* Ae system is essoidai to the definition of the 
object. Quite simply, a header is rq»laced by a pointer to a header, and die header 
infbnnation itself is located in some accessible location* The pouiter 1690 represents 
wtiaRver complete address infocmation is needed in order for an instrument or 
plication to locate the header infarmation, 1694, located in some binary stooge 
10 location (or database) 1692. The data sutiam 802 is whatever normal data that might be 
contained in a typical digital image file or binaiy record or software program or 
iriiatever. The pointer. 1690, can also be embedded steganographically as depicted, or 
alternatively, it can ONLY be assodatcd with the data stream steganogiaphicaliy . 

Fig. 61 begins to dissect Fig. 60, trying to depict a closer view of a practical MIO 
15 (**mastcr identity object"). Reverence is paid to the idea that common data objects and 
common data formats will be used for some time to come, and in many ways the MIO 
mformarion will be appended. Thus we start with a traditional header 1700 and die 
associated data stream 802. The MIO points is now split out into three pieces, a ^^tag"* 
which identifies the following information as a pointer to a remote database. Hie 
20 poinrer is now conveniaidy split into a pointer to a pointer 1704, effectively giving the 
address of some database in a sea of potential databases, while pointer 1706 is the 
ultimate address on ths found database pointed to by 1704. Indeed, 1704 could also be 
split cut, just as in surface mail type addressing. 

Hg. 61 also indicates that multiple layers of MIO mformation can be added to an 
26 object One example of this would be that an organization would track and account for 
all bicomhig and outgoing data objects and wish to provide a pointer to that given 
objects lifetime withm the organizations data network, yrt respect any previous MIO 
information that tiie creator or distributor of a data object already attached to diat (*}ecL 
There are numerous odier examples, including derivative work applications or situations 
30 where a data object evolves. 

Fig. 62 takes a closer look at the connectivity componoits of an MIO system, 
looiced at through the eyes of an instrument or application needing to retrieve the master 
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vkaxay infomiatioa for a given MIO-enabled data object An arbitcaiy data object 17 10 
enters a highly geneticized ''node'* 1711 Typically, this node might be an '*open file"* 
procedure in a software q^fdication, or it might be a ''seaxch for all im^e files'' 
operation initiated in a computefs operating system, or it might be a digital receiver in a 
tdevisionset 

The node 1712 is meant to represent the place where a given data object enters a 
given instnunent or application and is scanned for MIO traits. The node 1712's output 
is simply that the data object either is or is not an MIO type object If it discovers an 
MIO object as in 1716. it must nwtt ask the simple question of whether or not die given 
instruments cares or not, essentially asking whoher or not to go fetch the information 
pointed to by the embedded pointer. If it is not configored to do so, it goes on with its 
woric, 1718. If h is configured to go fetch the associated remote information (1720), it 
then singly goes Ihrou^ a basic address resolution process to locate the master 
identity. A first low level search might be qqrical wherein the "master identity" (which 
we have amply explained might actually be a search for further instmctians rather than 
only ''looking for a name") is potentially found on a user's own desktop compute or 
local instmmrat This might be as low level as a call to some memory location in the 
CPU sysian 1724, perhaps retrieving and therefor instigating instnicticms to prepare for 
video inforraatitm to follow. Or it might be a pointer 1726 to some location in a display 
device, perhaps as a precursor to understanding the current display settings that a user is 
operating under. 

The need for multiple layers of MIO infOTmatiw) as depicted in Hg. 61 becomes 
more zpparmx in these examples as well, in that the lower level retrieving wUl often be 
precursors to more advanced actions. Likewise, an address might be found in the 
operating system 1728, ROM, RAM, hard drives, etc. (1730). A token indication of a 
firewall 1732 is depicted showing that some addresses will only be accessible to a 
*'local object" by design. Finally, pertiaps die more common use of true "identity" will 
come from a route to a broader network 1734. Depicted is a certain remote database 
1736 where the given data object might have its master identity. These two basic ideas 
of "local" identity and "remote" identity are depicted below as 1738 and 1740, but it 
should be appreciated that tiiese can become quite bhnry in their distinctions. 

It should be understood that one object of this invention is that, for many 
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practical systeiDS* the locating process of itnding and retrieving the master identity 
mfonnatioB or instructicms is meant to tate place in realtinie, thai is, is typicaOy less 
than a second or two ideally, and no more than vsa seconds or twenty seconds even in 
case where there is in the muldpies of kilobytes to retrieve. This land of time 
6 performance is generally possible today on dlie internet even with sub- lOQEbit per 

sec(md modems, though less than a second is generally oat of reach of cuixent modem- 
based systems using standard phone lines and standard internet communicadons 
protocols. 

Fig. 63 traces out a generic process Qow within an instrument or application with 

10 a slightly different twist than depicted in Fig. 62. Again» any device capable of 

processing a data object reads said object in, and asks die sunple question of 
whether or not remote associated mfonnatiott exists for that object 1152, If not, fme« 
I7S4, go about your way. If so, it asks should I care, 1758, If not (1760, go about your 
way. If so, 1762, go fetch the xnformadon (or scries of infonnation if there is recursive 

15 MIO information). Box 1766 then explicitly shows that the user's device win still effect 
what happens even after the remote data is retrieved 

The permutadotvs and combinadons of operations from this point fiorward are as 
broad as general confuting principles permit As one tmy example, the user may have 
configured the device such that if the retrieved information does not match certain '*key 

20 wards" that the user has chosen, then don't even bother displaying an image (assuraing 
that is the data object in question). Essentially, the retrieved information bom 1764 and 
the user configuration 1766 set up a generally unlimited set of new instructions to the 
device. Another example might be that the user configuration 1766 says to go ahead 
and follow whatever instructions come back with the retrieved infonnation, such as 

85 "launch a browser and follow an URL to some JAVA web site and don't mind sending 
out my credit card infonnation if they ask'*. This is deliberately melodramatic to 
illustrate that the basic branching at 1770 is as broad as general computing principles 
permit 

Fig. 64 briefly illustrates that both embedded information as well as pointed-to 
30 information can actually sequences of in^ructions. including decision branched 

instracti(His. not merely singular instructions or singular blocks of passive descriptive 
information. This idea is encapsulated in the word "script'' as is gensrally well kxwwn 
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in the art of computing. 

Hg.65 essentiany depicts an MIOsysim operated b'^v^ 
previoQs few figures. In general, the idea here is that the master identity warns to know 
and collect all migraticii activities of its associaced data object^ and the system depicted 
io Fig. 65 15 essendaliy a way to do this. Again, we see a generic data object 1820 
entering some goienc ai^licatim or instnanent 1822, here cefeiied to as a '^reporting 
node''. The first question still is ""are you an MIOr 1824. 1826 alludes to die idea of 
reporting noa*hits to some kind of master MIO system database 1830, a database which 
might track the broad traffic patterns of MIO object vs. Noi-MIO objects. If the node 
finds an MIO» the next logical question is *'may" I send a message to its master identity 
in order to inform ibe master idendty (bat this object passed diis location at this 
particular time? (1834). This is tmpoitant since there will be many sitnatirais due to 
privacy and security considerations where such reporting would be intolerable 1834. 
However, diere may also be many simadons where diis reporting bfcoaatiou is 
valuable to all users of MIO based systems, and a "count" message wiH be sent to die 
master idairity location, wh»e locating die master identity follows previous 
descripdons- Details of how die master idendty cdlects messages arc left silent here as 
this form of data collection technology is readily available in today's marketplace and 
conq)uting systems, where uitra-murdane methods such as sending a self-contained e- 
mail to some e-mail address might even suffice as a mcUiodology of coundng (horribly 
ine£5cient but indicative of the vast arrays of mediods to have a master Identity 
database keep track of hundreds if not millions of copes of "itself). 

In view of the wide variety of implementations and applications to which the 
principles of this technology can be put, it should be s^parem diat the detailed 
embodiments are iilustradve only and in no way limit die scope of my invention. I 
claim as my invention all such embodiments as come within the scope and spirit of the 
fdlowing claims and equivalents thereto. 
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CLAIMS 

1 . A muld-coinputer system including a network for embedding asd 
reading a watermark, che system comprising: 

5 a first digital eiecihcal computer system comprising a fust digital dectrical 

computer electrically connected to a first input device, to a first omput device, and to a 
first mcxnoiy storing a plurality of creator idoitifiers and creator contact data 
corresponding to each of the creator identifiers; 

a second digital electrical coctq>uter system comprising a second digital electrical 

10 computer electrically connected to a second input device and to a second output device, 
the second digital electrical computer being programmed to embed a watermark in a 
digital |diotogra|Aic image, the watermark inchidtng one of die plurality of creator 
idemifim; 

a third digital electrical computer system comprising a third digital electrical 
ifi computer electrically connected to a third input device and to a third output device, the 
third digital electrical conqmter being programmed to read the watermaric in the digital 
photographic image to reveal the one of the ploiality of creator identifiers; 

a network for communicating the revealed one of the plurality of creator 
tdendfim to the fint digital electrical computer to obtain the creator contact data 
20 coiTcspondmg to the one of the plurality of creator identifies form the mcrooiy. 

2. The system of claim 1. wherein: 

the second digital electrical computer is programmed to automatically detect for 
a watermaric when an image is first examined by the second digital electrical computer. 

3. The system of claim 1, wherein: 

25 the second digital electrical computer is programmed to selectably detect for a 

watermaric when an image is examined by the second digital electrical computer. 

4. The system of claim I, wherein: 
the network includes the Internet; 

the watermark includes mformation idcnti^ymg a Worid Wide Web site; and 
30 wherein 

the third digital electrical computer system is programmed to load a World Wide 
Web browser and cwmect to the Worid Wide Web site in response to the revealed one 
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of the piurali^ of creator identifieis. 

5. The system of claim 1, wheiein: 

the wateimark includes extended data iocluding at lease one member from the 
group consistiRg of an organization idmtifier, a tiansactian idoitifier, and an item 
identifier. 

6. In a television set, audio stereo system, computing system, other 
system enable of receiving, storing, transmittmg, or processing m audio or visual 
signal, an improvement cominising: 

means for adding to an initial audio or visual signal an additional digital 
information associated with the initial signal, 

the additi(»ia} information containing, in whole or in part, an identifik;ation 
number which is uniquely assigned to the individual initial signal or copies thereof, 

wherein the identification number represents an address to a location in a remote 
database wherein a prestored set of information associated with the initial signal is 
contamed, and 

means coupled to the sjrstem containing a processor responsive to basic operating 
instructions for enabling the reading of the additional information and enabling the 
realtime retrieving from die remote database a subset of the prestored information via a 
communications link with the remote site. 

7. The system of claim 6, wherein the prestored set of information 
comprises identification and description information about the signal or an object 
contained in or r^resented by the signal. 

8. The system of claim 6» wherein die prestored set of information 
comprises a set of machine instructions executable to perform a task relative to the 
initial signal or about an €d)ject contained in the initial signal. 

9. The system of claim 6, wherein the jnestared set of information 
comprises a menu of options or commands. 

10. The sysrem of claim 1, including means for steganographically 
embedding the additional information in (he initial signal. 

1 1 . The system of claim 1 , wherein there also exists a second number as part 
of the additional information, wherein the second number is the address or identifier of 
the database itself to which the first identification points inside, in the context of 
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multiple databases serving a multitude of andio and visual signals. 

12. Hie system of claim 4, &st and second numbers steganpgraphicaUy 
embedded in the initial signal. 

13. Ibe system of claim 1, wherein there is additionally an instnunent or 
receiving stadon which numitors a multqple number of audio/visual public channels 
searching and recording the ifientification numbers and lepoidng sitings. 

14. In an image processing method that includes steganographically 
decoding an mput two-dunension^ image to extract a multi-bit code therein, the image 
comprising a two-dimensional array of pixels, an improvement comprising: 

cransforming the image into the spatial £cequency domain; 

pattern matching the transformed image so spatial frequencies obtained by said 
transfonning step coincide with reference spadal frequencies, to thereby effect 
registradon cf the transformed miage; 

inverse-transforming the transformed image to yidd a registered image; 

identifying, m the registered image, a plurality of regions that encode a first 
control bit, said regions being distributed through the registered image in a regular 
array; 

perfarming a statistical analysis over at lease said plurality of regions to 
determine whether the first control bit has first or second values; 

if said control fait has the first value, peifoiming a first decodmg process on the 
image to extract the code therefrom; and 

if said control bit has the second value, performing a second decoding process on 
the hnage to extract the code therefrom, the second decoding process bemg different 
than the first 
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DISTRIBUTION LOT: REGION 14 
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PSEUDO-RANDOM MASTER SNOWY IMAGE 
(SCALED DOWN AND ADDED TO FRAME 121 83) 
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96 BIT LEADER STRING. 820 



"SHADOW CHANNEL", 828 
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QUEST FOR MOSAICED KNOT PAHERNS WHICH 'COVER" AND 
ARE COEXTENSIVE WITH ORIGINAL IMAGE; 
ALL ELEMENTAL KNOT PATTERNS CAN CONVEY THE SAME ^ 
INFORMATION, SUCH AS A SIGNATURE, OR EACH CAN CONVEY A 
NEW MESSAGE IN A STEGANOGRAPHIC SENSE 
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2-D BRIGHTNESS OF PHASE-ONLY RLTERED 

RING IS SIMILAR TO THE ABOVE BRIGHTNESS PATTERN 

ROTATED ABOUT CENTRAL POINT OF RING: 
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EXAMPLE OF HOW MANY ELEMENTARY BUMPS, 900, WOULD BE ASSIGNED 
LOCATIONS IN AN IMAGE, AND THOSE LOCATIONS WOULD BE ASSOCIATED 
WITH A CORRESPONDING BIT PLANE IN THE N-BIT WORD. HERE TAKEN 
AS N=8 WITH INDEXES OF 0-7. ONE LXATION, ASSOCIATED WITH BIT 
PLANE "5". HAS THE OVERLAY OF THE BUMP PROFILE DEPICTED. 



wo 97/43736 



PCTAIS97/D8351 



16/55 



940 




FIG. 22 



DIGITAL IMAGE TAKEN 
OF CUSTOMER 




COMPUTER 
PROCESSES NEW 
ENCODED IMAGE 



^952 

Viirlni liVniv". 




PRINTS 
PROCESSED 
IMAGE ONTO 
PLASTIC CARD 



•954 



wo 97/43736 



FCTfQS9HmSl 



17/55 



FIG. 23 





CABLE, 964, 
TO DATA LINE, 
966 



/ 



CONTAINS RUDIMENTARY OPTICAL SCANNER, 
MEMORY BUFFERS, COMMUNICATIONS DEVICES. 
AND MICROPROCESSOR 



\ 



CONSUMER MERELY PLACES CARD INTO WINDOW 
AND CAN, AT THEIR PREARRANGED OPTION, EITHER 
TYPE IN A PERSONAL IDENTIFICATION NUMBER 
(PIN, FOR ADDED SECURITY) OR NOT THE TRANSACTION 
S APPROVED OR DISAPPROVED WITHIN SECONDS. 
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ORIGINAL DIGITAL IMAGE WITH 
BARCODE AND FIDUCIALS 
ADDED 
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COMPUTER GENERATES MASTER 
SNOWY IMAGE WHICH IS 
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ORIGINAL IMAGE AT LEFT 




COMBINED TO FORM PERSONAL CASH CARD 
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FIG. 26 

THE NEGLIGIBLE-FRAUD CASH CARD SYSTEM 




A BASIC FOUNDATION OF THE CASH CARD SYSTEM IS A 24-HOUR 
INFORMATION NETWORK, WHERE BOTH THE STATIONS WHICH CREATE 
THE PHYSICAL CASH CARDS, 950, AND THE POINT-OF-SALES, 984, 
ARE ALL HOOKED UP TO THE SAME NETWORK CONTINUOUSLY 
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FIG. 27B 
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EMBEDDED SIGNAL 
POWER SPECTRUM, 
1002 



UV PLANE, 1000 



TYPICAL POWER SPECTRUM 
OF ARBITRARY IMAGE, 
1004 



NON-HARMONIC SPATIAL FREQUENCIES ALONG THE 
45 DEGREE AXES, GIVING RISE TO A WEAVE-LIKE 
CROSS-HATCHING PATTERN IN THE SPATIAL DOMAIN 
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EMBEDDED SIGNAL 
POWER SPECTRUM, 
1006 




UV PLANE, 1000 



TYPICAL POWER SPECTRUM 
OF ARBITRARY IMAGE, 
1004 



NON-HARMONIC CONCENTRIC CIRCLES IN UV PLANE, 
WHERE PHASE HOPS QUASl-RANDOMLY ALONG EACH 
CIRCLE. GIVING RISE TO PSEUDO RANDOM LOOKING 
PATTERNS IN THE SPATIAL DOMAIN 
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PHASE OF SPATIAL 
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THIRD CONCENTRIC RING. 
1016 
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POWER PROFILE ALONG ANGLE A, AS NORMALIZED BY 
ITS OWN MOVING AVERAGE; ONLY A MINIMAL AMOUNT 
EXCEEDS THRESHOLD, GIVING A SMALL INTEGRATED VALUE 
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POWER PROFILE ALONG ANGLE B. AS NORMALIZED BY 
ITS OWN MOVING AVERAGE; THIS FINDS STRONG ENERGY 
ABOVE THE THRESHOLD 
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SCALE = A; ADD ALL POWER VALUES AT THE 
"KNOWN" FREQUENCIES". 1042 
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SCALE = B: ADD ALL POWER VALUES AT THE 
"KNOWN" FREQUENCIES", 1044 
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•SCALED-KERNEL" BASED MATCHED RLTER; PEAK IS 
WHERE THE SCALE OF THE SUBLIMINAL GRID WAS 
FOUND, 1046 
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FIG. 36A 
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ARBITRARY ORIGINAL IMAGE 
IN WHICH SUBUMINAL 
GRATICULES MAY HAVE BEEN PUCED 




■COLUMN SCAN" 
IS APPLIED ALONG A 
GIVEN ANGLE THROUGH 
THE CENTER OF THE 
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FIG. 37 

PROCESS STEPS 

1. SCAN IN PHOTOGRAPH 

2. 2D FfT 

3. GENERATE 2D POWER SPECTRUM, FILTER WITH E.G. 
3X3 BLURRING KERNEL 

4 STEP ANGLES FROM 0 DEGREES THROUGH 90 (1/2 DEG) 

5 GENERATE NORMALIZED VECTOR, WITH POWER VALUE 
AS NUMERATOR, AND MOVING AVERAGED POWER 
VALUE AS DENOMINATOR 

6 INTEGRATE VALUES AS SOME THRESHOLD. GIVING 
A SINGLE INTEGRATED VALUE FOR THIS ANGLE 

7. END STEP ON ANGLES 

8 FIND TOP ONE OR TWO OR THREE "PEAKS" FROM THE 
ANGLES IN LOOP 4, THEN FOR EACH PEAK... 

9 STEP SCALE FROM 25%'TO 400%, STEP - 1 .01 

10 ADD THE NORMAUZED POWER VALUES CORRESPONDING 
TO THE 'N' SCALED FREQUENCIES OF STANDARD 

11 . KEEP TRACK OF HIGHEST VALUE IN LOOP 

1 2. END LOOP 9 AND 8, DETERMINE HIGHEST VALUE 

1 3. ROTATION AND SCALE NOW FOUND 

1 4. PERFORM TRADITIONAL MATCHED FILTER TO 
FIND EXACT SPATIAL OFFSET 

15 PERFORM ANY "RNE TUNING" TO PRECISELY 
DETERMINE ROTATION, SCALE, OFFSET 
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